Folk theorem (game theory)
Folk theorem (game theory)
In game-theory, folk theorems are a class of theorems describing an abundance of nash-equilibrium payoff profiles in repeated games . The original Folk Theorem concerned the payoffs of all the Nash equilibria of an infinitely repeated game. This result was called the Folk Theorem because it was widely known among game theorists in the 1950s, even though no one had published it. Friedman's (1971) Theorem concerns the payoffs of certain subgame-perfect Nash equilibria (SPE) of an infinitely repeated game, and so strengthens the original Folk Theorem by using a stronger equilibrium concept: subgame-perfect Nash equilibria rather than Nash equilibria.
The Folk Theorem suggests that if the players are patient enough and far-sighted (i.e. if the discount factor <math> \delta \to 1 </math>), then repeated interaction can result in virtually any average payoff in an SPE equilibrium. "Virtually any" is here technically defined as "feasible" and "individually rational".
Setup and definitions We start with a basic game, also known as the stage game, which is an *n-*player game. In this game, each player has finitely many actions to choose from, and they make their choices simultaneously and without knowledge of the other player's choices. The collective choices of the players leads to a payoff profile, i.e. to a payoff for each of the players. The mapping from collective choices to payoff profiles is known to the players, and each player aims to maximize their payoff. If the collective choice is denoted by x, the payoff that player i receives, also known as player is utility*, will be denoted by <math>u_i(x)</math>.
We then consider a repetition of this stage game, finitely or infinitely many times. In each repetition, each player chooses one of their stage game options, and when making that choice, they may take into account the choices of the other players in the prior iterations. In this repeated game, a strategy for one of the players is a deterministic rule that specifies the player's choice in each iteration of the stage game, based on all other player's choices in the prior iterations. A choice of strategy for each of the players is a strategy profile, and it leads to a payout profile for the repeated game. There are a number of different ways such a strategy profile can be translated into a payout profile, outlined below.
Any nash-equilibrium payoff profile of a repeated game must satisfy two properties:
Individual rationality: the payoff must weakly dominate the minmax payoff profile of the constituent stage game. That is, the equilibrium payoff of each player must be at least as large as the minmax payoff of that player. This is because a player achieving less than their minmax payoff always has incentive to deviate by simply playing their minmax strategy at every history. #Feasibility: the payoff must be a convex combination of possible payoff profiles of the stage game. This is because the payoff in a repeated game is just a weighted average of payoffs in the basic games.
Folk theorems are partially converse claims: they say that, under certain conditions (which are different in each folk theorem), every payoff profile that is both individually rational and feasible can be realized as a Nash equilibrium payoff profile of the repeated game.
There are various folk theorems; some relate to finitely-repeated games while others relate to infinitely-repeated games.
Infinitely-repeated games without discounting In the undiscounted model, the players are patient. They do not differentiate between utilities in different time periods. Hence, their utility in the repeated game is represented by the sum of utilities in the basic games.
When the game is infinite, a common model for the utility in the infinitely-repeated game is the limit inferior of mean utility: If the game results in a path of outcomes <math>x_t</math>, where <math>x_t</math> denotes the collective choices of the players at iteration t (t=0,1,2,...), player i utility is defined as
::<math>U_i = \liminf_{T\to \infty} \frac{1}{T} \sum_{t=0}^T u_i(x_t),</math> where <math>u_i</math> is the basic-game utility function of player i.
An infinitely-repeated game without discounting is often called a "supergame".
The folk theorem in this case is very simple and contains no pre-conditions: every individually rational and feasible payoff profile in the basic game is a Nash equilibrium payoff profile in the repeated game.
The proof employs what is called a grim or [[grim-trigger]] The punishment should not last forever; it should last only a finite time which is sufficient to wipe out the gains from deviation. After that, the other players should return to the equilibrium path.
The limit-of-means criterion ensures that any finite-time punishment has no effect on the final outcome. Hence, limited-time punishment is a subgame-perfect equilibrium.
- Coalition subgame-perfect equilibria: With the limit-of-means criterion, a payoff profile is attainable in coalition-Nash-equilibrium or in coalition-subgame-perfect-equilibrium, if-and-only-if it is Pareto efficient and weakly-coalition-individually-rational.
Overtaking Some authors claim that the limit-of-means criterion is unrealistic, because it implies that utilities in any finite time-span contribute 0 to the total utility. However, if the utilities in any finite time-span contribute a positive value, and the value is undiscounted, then it is impossible to attribute a finite numeric utility to an infinite outcome sequence. A possible solution to this problem is that, instead of defining a numeric utility for each infinite outcome sequence, we just define the preference relation between two infinite sequences. We say that agent <math>i</math> (strictly) prefers the sequence of outcomes <math>y_t</math> over the sequence <math>x_t</math>, if:
::<math>\liminf_{T\to \infty} \sum_{t=0}^T ( u_i(y_t) - u_i(x_t)) > 0</math>
For example, consider the sequences <math>u_i(x)=(0,0,0,0,\ldots)</math> and <math>u_i(y)=(-1,2,0,0,\ldots)</math>. According to the limit-of-means criterion, they provide the same utility to player i, but according to the overtaking criterion, <math>y</math> is better than <math>x</math> for player i. See overtaking criterion for more information.
The folk theorems with the overtaking criterion are slightly weaker than with the limit-of-means criterion. Only outcomes that are strictly individually rational, can be attained in Nash equilibrium. This is because, if an agent deviates, he gains in the short run, and this gain can be wiped out only if the punishment gives the deviator strictly less utility than the agreement path. The following folk theorems are known for the overtaking criterion:
- Strict stationary equilibria:
- **Strict stationary subgame-perfect equilibria*': it can be attained in strict-stationary-coalition-equilibrium.
Infinitely-repeated games with discounting Assume that the payoff of a player in an infinitely repeated game is given by the average discounted criterion with discount factor 0 < δ < 1:
:<math>U_i = (1-\delta) \sum_{t \geq 0} \delta^t u_i(x_t),</math> The discount factor indicates how patient the players are. The factor <math>(1-\delta)</math> is introduced so that the payoff remain bounded when <math>\delta\rightarrow 1</math>.
The folk theorem in this case requires that the payoff profile in the repeated game strictly dominates the minmax payoff profile (i.e., each player receives strictly more than the minmax payoff).
Let a be a strategy profile of the stage game with payoff profile u which strictly dominates the minmax payoff profile. One can define a Nash equilibrium of the game with u as resulting payoff profile as follows:
:1. All players start by playing a and continue to play a if no deviation occurs.
:2. If any one player, say player i, deviated, play the strategy profile m which minmaxes i forever after.
:3. Ignore multilateral deviations.
If player i gets ε more than his minmax payoff each stage by following 1, then the potential loss from punishment is
:<math>\frac{1}{1-\delta} \varepsilon.</math>
If δ is close to 1, this outweighs any finite one-stage gain, making the strategy a Nash equilibrium.
An alternative statement of this folk theorem In considering, for instance, oligopoly behavior, the folk theorem does not tell the economist what firms will do, but rather that cost and demand functions are not sufficient for a general theory of oligopoly, and the economists must include the context within which oligopolies operate in their theory. The practical consequence of this is that no efficient (polynomial-time) algorithm is known that computes the strategies required by folk theorems in the general case.
Summary of folk theorems The following table compares various folk theorems in several aspects: * Horizon – whether the stage game is repeated finitely or infinitely many times. * Utilities – how the utility of a player in the repeated game is determined from the player's utilities in the stage game iterations. Conditions on G* (the stage game) – whether there are any technical conditions that should hold in the one-shot game in order for the theorem to work. Conditions on x* (the target payoff vector of the repeated game) – whether the theorem works for any individually rational and feasible payoff vector, or only on a subset of these vectors. * Equilibrium type – if all conditions are met, what kind of equilibrium is guaranteed by the theorem – Nash or Subgame-perfect? * Punishment type – what kind of punishment strategy is used to deter players from deviating?
{| class="wikitable sortable" |- ! Published by !! Horizon !! Utilities !! Conditions on G !! Conditions on x !! Guarantee !! Equilibrium type !! Punishment type |- | Benoit& Krishna || Finite (<math>T</math>) || Arithmetic mean || For every player there is an equilibrium payoff strictly better than minimax. || None || For all <math>\varepsilon>0</math> there is <math>T_0\in \mathbb{N}</math> such that, if <math>T\geq T_0</math>, for every <math>x</math> there is equilibrium with payoff <math>\varepsilon</math>-close to <math>x</math>. || Nash || |- | Aumann& Shapley || Infinite || Limit of means || None || None || Payoff exactly <math>x</math>. || Subgame-perfect || Limited-time punishment. || Infinite || Sum with discount <math>\delta</math> || Correlated mixed strategies are allowed. || Strictly above minimax. || When <math>\delta</math> is sufficiently near 1, there is an equilibrium with payoff exactly <math>x</math>. || Nash || Grim |- | Fudenberg& Maskin
|| Strictly above minimax. || For all <math>x</math> there is <math>\delta_0<1</math> such that, if <math>\delta\geq \delta_0</math>, there is equilibrium with payoff exactly <math>x</math>. || Subgame-perfect || Rewarding the punishers.