Von Neumann–Morgenstern utility theorem
Von Neumann–Morgenstern utility theorem
In decision theory, the von Neumann–Morgenstern (VNM) utility theorem demonstrates that rational choice under uncertainty involves making decisions that take the form of maximizing the expected value of some cardinal utility function. The theorem forms the foundation of expected utility theory.
In 1947, john-von-neumann and oskar-morgenstern proved that any individual whose preferences satisfied four axioms has a utility function, where such an individual's preferences can be represented on an interval scale and the individual will always prefer actions that maximize expected utility. That is, they proved that an agent is (VNM-)rational if and only if there exists a real-valued function u defined by possible outcomes such that every preference of the agent is characterized by maximizing the expected value of u, which can then be defined as the agent's VNM-utility (it is unique up to positive affine transformations i.e. adding a constant and multiplying by a positive scalar). No claim is made that the agent has a "conscious desire" to maximize u, only that u exists.
VNM-utility is a decision utility in that it is used to describe decisions. It is related, but not necessarily equivalent, to the utility of Bentham's utilitarianism.
Set-up In the theorem, an individual agent is faced with options called lotteries. Given some mutually exclusive outcomes, a lottery is a scenario where each outcome will happen with a given probability, all probabilities summing to one. For example, for two outcomes A and B,
::<math>L = 0.25A + 0.75B </math>
denotes a scenario where P(A) = 25% is the probability of A occurring and P(B) = 75% (and exactly one of them will occur). More generally, for a lottery with many possible outcomes Ai, we write:
::<math> L = \sum p_i A_i, </math>
with the sum of the <math>p_i</math>s equal to 1.
The outcomes in a lottery can themselves be lotteries between other outcomes, and the expanded expression is considered an equivalent lottery: 0.5(0.5A + 0.5B) + 0.5C = 0.25A + 0.25B + 0.50C.
If lottery M is preferred over lottery L, we write <math>M \succ L</math>, or equivalently, <math>L \prec M</math>. If the agent is indifferent between L and M, we write the indifference relation <math>L\sim M.</math> If M is either preferred over or viewed with indifference relative to L, we write <math>L \preceq M.</math>
The axioms The four axioms of VNM-rationality are completeness, transitivity, continuity, and independence. These axioms, apart from continuity, are often justified using the Dutch book theorems (whereas continuity is used to set aside lexicographic or infinitesimal utilities).
Completeness assumes that an individual has well defined preferences:
:Axiom 1 (Completeness) For any lotteries <math>L</math> and <math>M</math>, either <math>\, L\succeq M</math> or <math>\, M\succeq L</math>.
(the individual must express some preference or indifference). Note that this implies reflexivity.
Transitivity assumes that preferences are consistent across any three options:
:Axiom 2 (Transitivity) If <math>\,L \succeq M</math> and <math>\,M \succeq N</math>, then <math>\,L \succeq N</math>.
Axiom 1 and Axiom 2 together can be restated as the statement that the individual's preference is a total preorder.
Continuity assumes that there is a "tipping point" between being better than and worse than a given middle option:
:Axiom 3 (Continuity): If <math>\,L \preceq M\preceq N</math>, then there exists a probability <math>\,p\in[0,1]</math> such that ::<math>\,pL + (1-p)N\, \sim \,M</math>
where the notation on the left side refers to a situation in which L is received with probability p and N is received with probability (1–p).
Instead of continuity, an alternative axiom can be assumed that does not involve a precise equality, called the Archimedean property.
:Axiom 4′ (Reduction of compound lotteries): For any lotteries <math>L, L', N, N'</math> and any <math>p, q \in [0,1]</math>,
:: <math>\text{if} \qquad L\sim qL'+(1-q)N',</math> :: <math>\text{then} \quad pL+(1-p)N \sim pqL'+ p(1-q)N' + (1-p)N.</math>
To see how Axiom 4 implies Axiom 4', interchange <math>p</math> and <math>1-p</math> and replace <math>N \to qL'+(1-q)N'</math> and <math>M \to N</math> in the expression in Axiom 4, and expand.
The theorem For any VNM-rational agent (i.e. satisfying axioms 1–4), there exists a function u which assigns to each outcome A a real number u(A) such that for any two lotteries,
::<math>L\prec M \qquad \mathrm{if \, and \, only \, if} \qquad E(u(L)) < E(u(M)), </math>
where E(u(L)), or more briefly Eu(L) is given by
::<math>Eu(p_1A_1 + \cdots + p_nA_n) = p_1u(A_1) + \cdots + p_nu(A_n). </math>
As such, u can be uniquely determined (up to adding a constant and multiplying by a positive scalar) by preferences between simple lotteries, meaning those of the form pA + (1 − p)B having only two outcomes. Conversely, the preferences of any agent acting to maximize the expectation of a function u will obey axioms 1–4. Such a function is called the agent's von Neumann–Morgenstern (VNM) utility.
Proof sketch <div id='construction'></div> The proof is constructive: it shows how the desired function <math>u</math> can be built. Here we outline the construction process for the case in which the number of sure outcomes is finite.
Suppose there are n sure outcomes, <math>A_1\dots A_n</math>. Note that every sure outcome can be seen as a lottery: it is a degenerate lottery in which the outcome is selected with probability 1. Hence, by the Completeness and Transitivity axioms, it is possible to order the outcomes from worst to best: :<math>A_1\preceq A_2\preceq \cdots \preceq A_n</math>
We assume that at least one of the inequalities is strict (otherwise the utility function is trivial—a constant). So <math>A_1\prec A_n</math>. We use these two extreme outcomes—the worst and the best—as the scaling unit of our utility function, and define:
:<math>u(A_1)=0</math> and <math>u(A_n)=1</math>
For every probability <math>p\in[0,1]</math>, define a lottery that selects the best outcome with probability <math>p</math> and the worst outcome otherwise: :<math>L(p) = p\cdot A_n + (1-p)\cdot A_1</math> Note that <math>L(0)\sim A_1</math> and <math>L(1)\sim A_n</math>.
By the Continuity axiom, for every sure outcome <math>A_i</math>, there is a probability <math>q_i</math> such that: :<math>L(q_i) \sim A_i</math>.
Then :<math>0 = q_1\leq q_2\leq \cdots \leq q_n = 1</math>.
For every <math>i</math>, the utility function for outcome <math>A_i</math> is defined as :<math>u(A_i)=q_i</math>
so the utility of every lottery <math>M=\sum_i p_i A_i </math> is the expectation of u: :<math>u(M) = u\left(\sum_i p_i A_i \right) = \sum_i p_i u(A_i) = \sum_i p_i q_i </math>
To see why this utility function is consistent with the given preferences, consider a lottery <math>M = \sum_i p_i A_i </math>, which selects outcome <math>A_i</math> with probability <math>p_i</math>. But, by our assumption, the decision maker is indifferent between the sure outcome <math>A_i</math> and the lottery <math>q_i\cdot A_n + (1-q_i)\cdot A_1</math>. So, by the Reduction axiom, he is indifferent between the lottery <math>M</math> and the following lottery: :<math>M' = \sum_i p_i [q_i\cdot A_n + (1-q_i)\cdot A_1] </math> :<math>M' = \left(\sum_i p_i q_i \right) \cdot A_n + \left(\sum_i p_i(1-q_i)\right)\cdot A_1</math> :<math>M' = u(M)\cdot A_n + (1-u(M))\cdot A_1</math> The lottery <math>M'</math> is, in effect, a lottery in which the best outcome is won with probability <math>u(M)</math>, and the worst outcome otherwise.
Hence, if <math>u(M)>u(L)</math>, a rational decision maker would prefer the lottery <math>M</math> over the lottery <math>L</math>, because it gives him a larger chance to win the best outcome.
Hence: ::<math>L\prec M \; </math> if and only if <math> E(u(L)) < E(u(M)).</math>
Reaction Von Neumann and Morgenstern anticipated surprise at the strength of their conclusion. But according to them, the reason their utility function works is that it is constructed precisely to fill the role of something whose expectation is maximized:
<blockquote> "Many economists will feel that we are assuming far too much ... Have we not shown too much? ... As far as we can see, our postulates [are] plausible ... We have practically defined numerical utility as being that thing for which the calculus of mathematical expectations is legitimate." – VNM 1953, § 3.1.1 p.16 and § 3.7.1 p. 28 in which he posits that rational behavior can be described as maximizing the expectation of a function u, which in particular need not be monetary-valued, thus accounting for risk aversion. This is the expected utility hypothesis. As stated, the hypothesis may appear to be a bold claim. The aim of the expected utility theorem is to provide "modest conditions" (i.e. axioms) describing when the expected utility hypothesis holds, which can be evaluated directly and intuitively:
<blockquote> "The axioms should not be too numerous, their system is to be as simple and transparent as possible, and each axiom should have an immediate intuitive meaning by which its appropriateness may be judged directly. In a situation like ours this last requirement is particularly vital, in spite of its vagueness: we want to make an intuitive concept amenable to mathematical treatment and to see as clearly as possible what hypotheses this requires." – VNM 1953 § 3.5.2, p. 25 </blockquote>
As such, claims that the expected utility hypothesis does not characterize rationality must reject one of the VNM axioms. A variety of generalized expected utility theories have arisen, most of which drop or relax the independence axiom.
Implications for ethics and moral philosophy Because the theorem assumes nothing about the nature of the possible outcomes of the gambles, they could be morally significant events, for instance involving the life, death, sickness, or health of others. By constructing/defining a suitable utility function, a von Neumann–Morgenstern rational agent is capable of taking such outcomes into account, sacrificing personal wealth or well-being for the benefit of the collective. In other words, both what is naturally perceived as "personal gain", and what is naturally perceived as "altruism", can be included in the VNM-utility function of a VNM-rational individual. Therefore, the full range of agent-focused to agent-neutral behaviors is possible with various VNM-utility functions.
If the utility of <math>N</math> is the same as that of <math>pM</math>, a von Neumann–Morgenstern rational agent must be indifferent between <math>1N</math> and <math>pM+(1-p)0</math>. An agent-focused von Neumann–Morgenstern rational agent therefore cannot favor more equal, or "fair", distributions of utility between its own possible future selves.
Distinctness from other notions of utility Some utilitarian moral theories are concerned with quantities called the "total utility" and "average utility" of collectives, and characterize morality in terms of favoring the utility or happiness of others with disregard for one's own. These notions can be related to, but are distinct from, VNM-utility: 1) VNM-utility is a decision utility*: it is that according to which one decides, and thus by definition cannot be something which one disregards. * 2) VNM-utility is not canonically additive across multiple individuals (see Limitations), so "total VNM-utility" and "average VNM-utility" are not immediately meaningful (some sort of normalization assumption is required).
The term E-utility for "experience utility" has been coined to refer to the types of "hedonistic" utility like that of Bentham's greatest happiness principle. Since morality affects decisions, a VNM-rational agent's morals will affect the definition of its own utility function (see above). Thus, the morality of a VNM-rational agent can be characterized by correlation of the agent's VNM-utility with the VNM-utility, E-utility, or "happiness" of others, among other means, but not by disregard for the agent's own VNM-utility, a contradiction in terms.
Limitations ### Nested gambling Since if L and M are lotteries, then pL + (1 − p)M is simply "expanded out" and considered a lottery itself, the VNM formalism ignores what may be experienced as "nested gambling". This is related to the Ellsberg problem where people choose to avoid the perception of risks about risks. Von Neumann and Morgenstern recognized this limitation:
<blockquote>"...concepts like a specific utility of gambling cannot be formulated free of contradiction on this level. This may seem to be a paradoxical assertion. But anybody who has seriously tried to axiomatize that elusive concept, will probably concur with it." – VNM 1953 § 3.7.1, p. 28.</blockquote>