1: \documentclass{format170x240multiauthor}
2: \usepackage[T1]{fontenc}
3: \usepackage{makeidx}
4: \makeindex
5: \usepackage{amsmath}
6: \usepackage{textcomp}% for upright mu (\textmu)
7: \usepackage{graphicx}
8: \usepackage{cite}
9: \usepackage{url}
10: \urlstyle{rm}% for List of Contributors
11:
12:
13: \oddsidemargin 0.0in
14: \evensidemargin 0.0in
15: \topmargin 0.2in
16:
17: \begin{document}
18:
19: \pagenumbering{arabic}
20:
21:
22:
23: \chapter{Stochastic evolutionary game dynamics}
24: \label{ch1}
25: \chapterauthor[]{Arne Traulsen and Christoph Hauert}
26:
27: %\subsection*
28: \vspace{-5cm}
29: {\sffamily To appear in {\bf ``Reviews of Nonlinear Dynamics and Complexity'' Vol. II}, Wiley-VCH, 2009, edited by H.-G. Schuster }
30: \vspace{6cm}
31:
32:
33:
34: \section{ Game theory and evolution}
35:
36: Modern game theory goes back to a series of papers by the mathematician John von Neumann in the 1920s. This program started a completely new branch of social sciences and applied mathematics.
37: This early work on game theory is summarized in the seminal book ``The Theory of Games and Economic Behavior'' by John von Neumann and Oskar Morgenstern \cite{neumann:1944ef}. Initially, game theory was primarily focused on cooperative game theory, which analyzes optimal strategies assuming that individuals stick to previous agreements.
38: In the 1950's, the focus shifted to non-cooperative games in which individuals act selfish to get the most out of an interaction. At that time, game theory had matured from a theoretical concept to a scientific field influencing political decision making, mainly in the context of the arms race of the cold war.
39:
40: The basic assumption was that individuals act rationally and take into account that their interaction partners know that their decisions are rational and vice versa. Based on a common utility function that individuals maximize, the actions of others can be predicted and the optimal strategy can be chosen. However, the underlying assumption of rationality is often unrealistic. Even in simple interactions between two individuals $A$ and $B$, it is difficult to imagine fully rational decision making,
41: as this often leads to an infinite iteration: $A$ thinks of $B$, who is thinking of $A$, who is thinking of $B$ and so on.
42:
43: One way to avoid this situation in economy is the idea of bounded rationality \cite{selten:1990le,gigerenzer:2002aa}. If the cost of acquiring and processing information is taken into account, individuals can no longer be assumed to do a fully rational analysis of the situation. Instead, their rationality is bounded and the way they make decisions is very different.
44: In this case the prediction of the action of others is usually no longer possible. Nonetheless, bounded rationality turned out to be a crucial concept to overcome the problematic rationality assumption in economics.
45:
46: A completely different way to deal with this problem in biology has been proposed by John Maynard Smith and George Price in the 1970s \cite{maynard-smith:1973to,maynard-smith:1982to}. They introduced the idea of evolutionary game theory.
47: The utility function (which is usually the payoff from the game) is identified with the reproductive fitness. In this way, successful strategies spread in the population and less successful strategies diminish. Two important aspects differ from the traditional setting of classical game theory:
48: (i) Evolutionary game theory always deals with populations of individuals instead of two (or few) players.
49: (ii) In contrast to the static classical game theory analysis, evolutionary game theory provides a natural way to introduce a dynamics into the system.
50:
51: There are two interpretations of evolutionary game dynamics. The first one is the traditional setting, in which strategies are encoded by the genome of individuals and successful types spread in the population due to their higher reproduction.
52: Examples from biology include the competition of different bacterial strains \cite{kerr:2002xg}, cooperation in virus populations \cite{turner:1999hp}, or the cyclic dominance of mating strategies in lizards \cite{sinervo:1996le,sinervo:2006aa}.
53: Biological reproduction selects successful strategies and does not require rational agents or other forms of cognitive abilities.
54:
55: The second interpretation is cultural evolution. In this setting, successful behaviors are copied by other individuals through imitation. Successful strategies
56: propagate through imitation and learning. Although individuals now have to make decisions, this is very different from the rational decisions in classical game theory. Instead of analyzing the situation in detail, the players just imitate those that are more successful. Such strategies are possible even with minimal cognitive premises. This approach is taken for comparisons between predictions of evolutionary game theory and behavioral studies.
57:
58: More recently, ideas of evolutionary game theory have been reintroduced to economics, where they are tackled with great mathematical rigor \cite{sandholm:2007bo}.
59:
60:
61:
62: \section{The replicator dynamics}
63:
64: Traditionally, evolutionary game dynamics is described for very large, unstructured populations.
65: In this case, a differential equation governs the evolution of the densities of the different strategies
66: \cite{zeeman:1980ze,taylor:1978wv,hofbauer:1998mm},
67: \begin{equation}
68: \dot x_i = x_i \left( \pi_i - \langle \pi \rangle \right).
69: \label{repdyn}
70: \end{equation}
71: Here, $x_i$ is the fraction of type $i$ in the population, $\pi_i$ is the
72: fitness of this type and $ \langle \pi \rangle $ is the average payoff in
73: the whole population. If the fitness of a type is above the average fitness in the population, its density will increase. If the fitness is below the average fitness, then
74: the corresponding density will decrease.
75: Each type $i$ has a fixed strategy.
76: If the fitness values $\pi_i$ are fixed, we speak of constant selection.
77: In general, $\pi_i $ depends on the composition of the population, i.e.\ on the fractions of all other strategies $x_j$.
78: Then, $ \langle \pi \rangle $ becomes quadratic in the fractions $x_j$. Therefore, the dynamics is nonlinear in general. Since the variables in the replicator equation represent the fractions of each strategy in the population, the natural coordinate system is a probability simplex, i.e., for 2 strategies, we have a line, for 3 strategies an equilateral triangle, for 4 strategies an equilateral tetrahedron and so on.
79:
80: As an example, let us consider the simplest possible games. These are so called two player normal form games or $2 \times 2$ games. Such games can be described by a payoff matrix of the following form
81: \begin{equation}
82: \bordermatrix{
83: & A & B \cr
84: A & a & b \cr
85: B & c & d \cr}.
86: \end{equation}
87: This is a way to specify the interactions of two types: If $A$ interacts with another $A$,
88: it obtains $a$ and $b$ if it interacts with $B$. Similarly, $B$ obtains $c$ from interactions with $A$ and $d$ from interactions with $B$. The payoffs are determined by the fraction of interactions with a given type. Since we have only two types, the population state is fully determined by $x=x_1=1-x_2$. The payoffs are then
89: $\pi_A = a \, x+b(1-x)$
90: and
91: $\pi_B = c \, x+d(1-x)$. This leads to the replicator equation
92: \begin{equation}
93: \dot x = x(1-x) \left[ (a-b-c+d) x + b-d \right].
94: \end{equation}
95: Apart from the trivial fixed points $x=0$ and $x=1$, the replicator equation can have a third fixed point $x^\ast$
96: for $a>c$ and $d>b$ or for $a<c$ and $d<b$,
97: \begin{equation}
98: x^{\ast} = \frac{d-b}{a-b-c+d}.
99: \end{equation}
100: We can distinguish four generic cases \cite{nowak:2004aa}, see Fig.~\ref{fig1}:
101: \begin{itemize}
102: \item {\bf Dominance.}
103: In this case, one strategy is always a better choice, regardless of the action of the opponent.
104: Either $A$ dominates $B$ ($a>c$ and $b>d$) or $B$ dominates $A$ ($a<c$ and $b<d$). In the first case, the fixed point at $x=1$ is stable and the fixed point at $x=0$ is unstable and vice versa in the latter case.
105:
106: \item {\bf Bistability.} This is the case for $a>c$ and $d>b$. The fixed points at $x=0$ and $x=1$ are stable, separated by an unstable fixed point $x^{\ast}$. The corresponding games are called coordination games.
107: What is the best strategy in such a game? As a first approximation, one can ask for the maximum payoff in the equilibrium. However, if the opponent is unreliable, one should also try to avoid large losses. This leads to the concept of risk dominance: The strategy that has a larger basin of attraction is called risk dominant. In our case, strategy $A$ is risk dominant for $a+b>c+d$ (or, equivalently $x^{\ast}< \frac{1}{2}$). For $a+b<c+d$ (or $x^{\ast}> \frac{1}{2}$), strategy $B$ is risk dominant.
108:
109: \item {\bf Coexistence.} For $a<c$ and $b>d$, there is a stable fixed point at $x^{\ast}$. Hence, the population becomes a stable mixture of $A$ and $B$ types. Both $x=0$ and $x=1$ are unstable fixed points.
110:
111: \item {\bf Neutrality.} For $a=c$ and $b=d$, the replicator dynamics predicts neutrally stable fixed points for all values of $x$. While this non-generic case is of limited interest in the replicator dynamics, neutral selection becomes an important reference case in the stochastic evolutionary dynamics of finite populations.
112: \end{itemize}
113:
114:
115:
116: \begin{figure}[t]
117: \def\capfrac{1}
118: \begin{center}
119: \includegraphics[width=1.0\textwidth]{Fig1.pdf}
120: \end{center}
121: \Caption{
122: The four
123: dynamical scenarios of evolutionary $2 \times 2$ games. The arrows
124: indicate the direction of selection, filled circles are stable fixed points and open circles unstable fixed points. In the neutral case, the entire line consists of neutrally stable fixed points. }
125: \label{fig1}
126: \end{figure}
127:
128:
129: In a genetic setting, the replicator equation is obtained when individuals reproduce at a rate proportional to their fitness.
130: In cultural settings, the replicator equation results from individuals imitating better performing actions with a probability proportional to the expected increase in the payoff.
131: Moreover, the replicator equations are intimately connected to the Lotka-Voltera equations describing predator-prey dynamics in theoretical ecology. In fact, a direct mathematical mapping from one framework to the other exists \cite{hofbauer:1998mm}.
132:
133: In general, replicator equations can exhibit very rich dynamics. However, in most cases, the location and stability of fixed points can be determined analytically. In addition, symmetry properties can lead to constants of motion. In this case, the dynamics follows closed orbits. In higher dimensions, the replicator equations can also exhibit deterministic chaos \cite{sato:2002le,sato:2003le,schuster:1995le}.
134: However, the replicator dynamics is just one particular variant of a deterministic selection dynamics.
135: If individuals switch to better strategies independent of the increase in the payoff, one obtains a
136: dynamics sometimes called ``imitate the better''.
137: Naturally, this would change the form of the differential equation \eqref{repdyn}.
138: The replicator equation and ``imitate the better'' are two examples of non-innovative selection dynamics, i.e. a strategy that goes extinct will never re-appear. However, in social models, individuals could also choose the ``best reply''
139: to the current composition of the population, even if such a strategy is not present in the population. This is an example of innovative game dynamics for which equally simple differential equations as \eqref{repdyn} can no longer be
140: defined.
141:
142: \section{Evolutionary games in finite populations}
143:
144: While replicator equations have very nice mathematical properties and provide many important insights into evolutionary game dynamics, they essentially describe the deterministic dynamics in an infinitely large population. In general, it is not a priori clear under which circumstances this is a good approximation of the dynamics in a realistic system which is finite and subject to fluctuations. One important source of fluctuations is internal noise arising from the finiteness of the population.
145: To answer these questions, we take the following approach: First, we describe evolutionary game dynamics as a stochastic process in a finite population. Then, we increase the population size and infer under which circumstances we recover the replicator dynamics. This also allows us to identify parameter combinations for which it is a good approximation. Moreover, we can
146: investigate whether other limits result in qualitatively different dynamics.
147:
148: \subsection{Stochastic evolutionary game dynamics}
149:
150: Evolutionary games in finite populations have been considered for a long time
151: in various fields such as theoretical ecology, behavioral economics or sociology.
152: For computer simulations of multi-agent systems, this is the only natural approach to model the dynamics.
153: In all cases, we have to specify the microscopic mechanisms which govern
154: the transmission of strategies from one individual to another.
155: Examples for such selection dynamics are
156: \begin{itemize}
157:
158: \item {\bf Pairwise comparison processes.}
159: In this class of processes, two individuals, a focal individual and a role model, are sampled at random from the population. The focal individual accepts the strategy of the role model with probability $p$, depending on a payoff comparison.
160: If both individuals have the same payoff, the focal individual randomizes between the two strategies. For example, the probability $p$ could be a linear function of the payoff difference \cite{traulsen:2005hp},
161: \begin{equation}
162: p=\frac{1}{2}+w \frac{\pi_f - \pi_r}{\Delta \pi}.
163: \label{fermieq}
164: \end{equation}
165: Here, $w$ ($0 \leq w \leq 1$) is the intensity of selection,
166: which specifies the importance of neutral drift compared to the selection dynamics.
167: This determines the noise intensity. The functions $\pi_f $ and $ \pi_r$ are the payoffs of the focal individual and the role model, respectively, and $\Delta \pi$ is the maximum payoff difference. For $ w \ll 1$, one speaks of "weak selection". Most importantly, the focal individual does not always switch to the better strategy - sometimes, it also adopts worse strategies.
168:
169: One common choice of a nonlinear function of the payoff difference for $p$ is the Fermi function from statistical mechanics, leading to
170: \begin{equation}
171: p=\left[1+e^{w(\pi_f - \pi_r)} \right]^{-1}.
172: \label{fermieq}
173: \end{equation}
174: Here, the intensity of selection
175: relates to an inverse temperature and can be any positive number. For weak selection $ w \ll 1$, the probability $p$ reduces to a linear function of the payoff difference.
176: For strong selection, $w \to \infty$, this process converges to the imitation dynamics.
177: In this case, $p$ becomes a step function being positive for $\pi_r > \pi_f $
178: and negative for $\pi_r < \pi_f $.
179: In this limit, only the order of the payoffs is important - an arbitrarily small difference in the payoffs leads to the same outcome.
180:
181:
182: \item {\bf Moran process.} The Moran process is a classical model of population genetics \cite{moran:1962ef} and has been transferred to game theory only recently \cite{nowak:2004pw,taylor:2004wv}. Here, one individual is chosen at random, but proportional to fitness.
183: This individual produces one identical offspring. To keep the number of individuals constant,
184: a randomly chosen individual is removed from the population before the offspring is added.
185: The Moran process represents a simple birth-death process.
186: To illustrate the selection mechanism, it is instructive to think of a roulette wheel in which the size of the different fields is proportional to the fitness. Fitness is a convex combination of a background fitness (usually set to one) and the payoff from the game, i.e. $f=1-w+w \pi$. Again, $w$ determines the intensity of selection. For $w=0$, selection is neutral and we have an undirected random walk. For $w \to 1$, fitness equals payoff. However, if the payoffs $\pi$ can become negative, there is a maximum intensity of selection, because the fitness has to be positive. This restriction can be overcome if fitness is defined as an exponential function of payoff,
187: $f=\exp \left[ w \pi \right]$. In this case, the selection intensity $w$ can take any positive value \cite{traulsen:2008aa}.
188:
189: \item {\bf Wright-Fisher process.}
190: The Wright Fisher process has its roots in population genetics as well.
191: In contrast to the selection dynamics in the Moran process, where only one individual reproduces at a time, the Wright-Fisher process reflects discrete generations. In every generation, each of the $N$ individuals produce a large number of offspring, proportional to their fitness. From this large offspring pool, a new generation again of size $N$ is randomly sampled.
192: Thus, the composition of the population can change much faster. In principle, the population could go back to a single ancestor in a single generation. This reflects the fact that the Wright-Fisher process is not a simple birth-death process, but a more general Markov process. Although it becomes very similar to the Moran process under weak selection, the fixation probabilities (see next section)
193: cannot be calculated exactly \cite{imhof:2006aa}.
194:
195: \end{itemize}
196: The replicator equation determines how the frequencies of strategies in the population change: For each point in state space, the direction of selection and its velocity is determined.
197: For finite populations, we have to deal with probabilities instead. Moreover, the state space is discrete. In this discretized space, we can calculate into which direction the system will evolve with what probability.
198: This imposes significant restrictions on analytically accessible results. However, the general properties can already be inferred from the simplest possible case, consisting of only two strategies.
199:
200: \subsection{Fixation probabilities}
201:
202: For non-innovative game dynamics, we have two absorbing states: either all individuals are of type $A$ or of type $B$.
203: One important determinant of the dynamics is given by the probabilities of fixation:
204: If a mutation leads to a new strategy, how likely is it that this individual takes over the entire population?
205:
206: For the sake of simplicity, let us focus on birth-death processes in which only a single individual reproduces at a time (we thus exclude processes as the Wright-Fisher process). We consider a population of size $N$ where the number of $A$ players is $j$ and the number of $B$ players is $N-j$.
207: The probability to increase the number of $A$ individuals from $j$ to $j+1$ is $T^+_j$. Similarly, $T_j^-$ is the probability to decrease $j$ by 1. Our goal is to calculate the fixation probabilities $\phi_j$, i.e.\ the probability that $j$ individuals of type $A$ succeed and take over the population.
208: For the absorbing states, we have
209: \begin{eqnarray}
210: \phi_0 = 0 \quad \hbox{and} \quad
211: \phi_N = 1.
212: \end{eqnarray}
213: For the intermediate states, the fixation probabilities are given by
214: \begin{equation}
215: \phi_j = T_j^- \phi_{j-1} + (1-T_j^--T_j^+) \phi_j + T_j^+ \phi_{j+1}.
216: \end{equation}
217: We can rearrange this to
218: \begin{equation}
219: 0 = -T_j^- \underbrace{(\phi_j-\phi_{j-1})}_{y_j} + T_j^+\underbrace{( \phi_{j+1} - \phi_{j})}_{y_{j+1}}.
220: \end{equation}
221: This equation can be written as a recursion for the differences between fixation probabilities,
222: $y_{j+1} = {\gamma_j} y_j$, where ${\gamma_j} = \frac{T_j^-}{T_j^+} $.
223: Using this iteration, we find
224: \begin{eqnarray}
225: y_1 &=& \phi_1 - \phi_0 = \phi_1\\
226: y_2 &=& \phi_2 -\phi_1 = \gamma_1 \phi_1 \\
227: & \vdots & \nonumber \\
228: y_k & = & \phi_k-\phi_{k-1} =\phi_1 \prod_{j=1}^{k-1} \gamma_j
229: \label{eqref12}
230: \\
231: & \vdots & \nonumber \\
232: y_N & = & \phi_N-\phi_{N-1} = \phi_1 \prod_{j=1}^{N-1} \gamma_j
233: \end{eqnarray}
234: As usual, the empty product is one, $ \prod_{j=1}^{0} \gamma_j=1$.
235: Let us now calculate the sum over all $y_j$. This sum is a ``telescope-sum'' and simplifies to
236: \begin{eqnarray}
237: \sum_{k=1}^N y_k = \phi_1 -\underbrace{\phi_0}_0+\phi_2 -\phi_1 +\phi_3-\phi_2 + \ldots +\underbrace{\phi_N}_{1}-\phi_{N-1} =1.
238: \label{eqref13}
239: \end{eqnarray}
240: Using Eqs.~\eqref{eqref12} and \eqref{eqref13}, we can finally calculate $\phi_1$,
241: \begin{eqnarray}
242: 1 = \sum_{k=1}^N y_k
243: = \sum_{k=1}^N \phi_1 \prod_{j=1}^{k-1} \gamma_j
244: = \phi_1 \left( 1+ \sum_{k=1}^{N-1} \prod_{j=1}^{k} \gamma_j \right)
245: \end{eqnarray}
246: Hence, the fixation probability of a single $A$ individual, $\phi_1$, is given by
247: \begin{equation}
248: \phi_1 = \frac{1}{1+ \sum_{k=1}^{N-1} \prod_{j=1}^{k} \gamma_j}
249: \label{start}
250: \end{equation}
251: For $T_j^- = T_j^+$, we have $\gamma_j=1$. Hence, all products are simply one and we find $\phi_1 = 1/N$. This is the case of neutral selection, where all individuals have the same fitness. Any random walk in which the probability to move to the left or to the right is identical for the transient states leads to the same result.
252:
253: So far, we have focused on
254: the fixation probability of a single mutant in a resident population, $\phi_1$. In general, the fixation probability $\phi_i$ is given by
255: \begin{eqnarray}
256: \phi_i
257: & = & \sum_{k=1}^{i} y_k \\
258: & = & \phi_1 \sum_{k=1}^{i} \prod_{j=1}^{k-1} \gamma_j \\
259: & = & \phi_1 \left( 1+ \sum_{k=1}^{i-1} \prod_{j=1}^{k} \gamma_j \right)\\
260: &=& \frac{1+ \sum_{k=1}^{i-1} \prod_{j=1}^{k} \gamma_j}{1+ \sum_{k=1}^{N-1} \prod_{j=1}^{k} \gamma_j}
261: \label{fixprob}
262: \end{eqnarray}
263: For neutral selection, we have $T^+_j=T^-_j$, which results in $\gamma_j=1$. In this case, the fixation probability reduces to $\phi_i = i/N$.
264:
265:
266: Formally, the fixation probability can be calculated in systems in which the replicator equation predicts coexistence, i.e.\ no fixation. However, it can also be shown that in these cases, the average time until fixation grows exponentially with increasing population size \cite{antal:2006aa} and increasing intensity of selection \cite{traulsen:2007cc}.
267:
268:
269: Often, the comparison between the probability that a single $A$ individual takes over a population of $N-1$ resident $B$ individuals, $\rho_A = \phi_1$, and the probability that a single $B$ individual takes over a population of $N-1$ resident $A$ individuals, $\rho_B$, is of interest in order to determine in which state the system spends more time
270: \cite{nowak:2006bo}. Formally, the probability $\rho_B$ is equal to the probability that $N-1$ individuals of type $A$ fail to take over a population in which there is just a single $B$ individual. Hence, we find
271: \begin{eqnarray}
272: \rho_B &=& 1 - \phi_{N-1} \\
273: & = & 1 - \frac{1+ \sum_{k=1}^{N-2} \prod_{j=1}^{k} \gamma_j}{1+ \sum_{k=1}^{N-1} \prod_{j=1}^{k} \gamma_j} \\
274: & = & \frac{1+ \sum_{k=1}^{N-1} \prod_{j=1}^{k} \gamma_j}{1+ \sum_{k=1}^{N-1} \prod_{j=1}^{k} \gamma_j}
275: - \frac{1+ \sum_{k=1}^{N-2} \prod_{j=1}^{k} \gamma_j}{1+ \sum_{k=1}^{N-1} \prod_{j=1}^{k} \gamma_j} \\
276: & = & \frac{\prod_{j=1}^{N-1} \gamma_j}{1+ \sum_{k=1}^{N-1} \prod_{j=1}^{k} \gamma_j} \\
277: & = & \rho_A {\prod_{j=1}^{N-1} \gamma_j}
278: \end{eqnarray}
279: Therefore, the ratio of the two fixation probabilities is given by
280: $
281: \frac{\rho_B}{\rho_A} = {\prod_{j=1}^{N-1} \gamma_j}.
282: $
283: If this product is smaller than 1, we have $\rho_B<\rho_A$, if it is larger than 1, we have
284: $\rho_B>\rho_A$.
285: For small mutation rates, $\rho_B<\rho_A$ means that the system
286: spends more time in the $A$ state, because less invasion attempts are necessary to reach fixation by $A$. The $A$ mutants have a higher probability to reach fixation in a $B$ population compared to $B$ mutants in an $A$ population.
287:
288:
289: \subsection{Fixation times}
290:
291: Another quantity of interest in evolutionary dynamics of finite population is the average time until fixation occurs \cite{ewens:2004qe}. For two strategies, three different fixation times are of interest:
292: \begin{itemize}
293: \item[(i)] The average time $t_j$ until either one of the two absorbing states, $A$ or $B$, is reached when starting from state $j$. This is the unconditional fixation time.
294: For weak selection, this time increases with the distance
295: between $j$ and the two absorbing boundaries.
296: \item[(ii)] The conditional fixation time $t_j^A$ specifies the average time it takes to reach the absorbing state $A$ when starting from state $j$, provided
297: that $A$ is ultimately reached.
298: The time $t_j^A$ increases with the distance between the states $j$ and $A$.
299: If fixation of strategy $A$ is almost certain,
300: $t_j^A$ is very similar to the unconditional fixation time $t$.
301: Of particular interest is $t_1^A$, or $t^A$ for short, which denotes the average fixation time of a single $A$ mutant in a resident $B$ population.
302: \item[(ii)] In analogy to $t^A_j$, $t^B_j$ represents the average time to reach the absorbing state $B$ when starting in state $j$ (i.e. with $j$ individuals of type $A$ and $N-j$ of type $B$), provided
303: that $B$ is ultimately reached.
304: $t^B_j$
305: increases with the distance
306: between state $j$ and $B$.
307: \end{itemize}
308:
309: \paragraph{Unconditional fixation time.}
310: The unconditional average fixation time $t_j$, starting from state $j$, is determined by
311: \begin{eqnarray}
312: t_j = 1+ T^-_j t_{j-1} +(1-T^-_j-T^+_j) t_j + T^+_j t_{j+1}.
313: \label{timeq0}
314: \end{eqnarray}
315: This equation can be rationalized as follows: In one time step (this results in the $1$), the process can either move to $j -1$, stay in $j$ or move to $j+1$. Then, the fixation time from that state matters. When we start in $0$ or in $N$, fixation has already occurred, thus $t_0=t_N=0$.
316: Eq.~\eqref{timeq0} can be written as
317: \begin{eqnarray}
318: \underbrace{t_{j+1}-t_j }_{z_{j+1}}= \gamma_j (\underbrace{ t_j-t_{j-1} }_{z_{j}})- \frac{1}{T_j^+}.
319: \label{timeq1}
320: \end{eqnarray}
321: With the notation ${\gamma_j} = \frac{T_j^-}{T_j^+} $ from above, iteration yields
322: \begin{eqnarray}
323: z_1 &=& t_1 - t_0 = t_1\\
324: z_2 &=& t_2 -t_1 = \gamma_1 t_1 - \frac{1}{T_1^+}\\
325: z_3 &=& t_3 -t_2 = \gamma_2 \gamma_1 t_1 - \frac{\gamma_2}{T_1^+} - \frac{1}{T_2^+}\\\
326: & \vdots & \nonumber \\
327: z_k & = & t_{k}-t_{k-1} =t_1 \prod_{m=1}^{k-1} \gamma_m - \sum_{l=1}^{k-1} \frac{1}{T_l^+} \prod_{m=l+1}^{k-1} \gamma_m
328: \end{eqnarray}
329: For the sum of the $z_k$, we find
330: \begin{eqnarray}
331: \sum_{k=j+1}^N z_k =
332: t_{j+1}-t_{j} + t_{j+2}-t_{j+1} + \ldots + \underbrace{t_N}_{=0}-t_{N-1} =
333: - t_{j}
334: \end{eqnarray}
335: In particular, we have for $j=1$
336: \begin{eqnarray}
337: t_1 = -\sum_{k=2}^N z_k
338: = - t_1 \sum_{k=1}^{N-1} \prod_{m=1}^{k} \gamma_m + \sum_{k=1}^{N-1} \sum_{l=1}^{k} \frac{1}{T_l^+} \prod_{m=l+1}^{k} \gamma_m.
339: \end{eqnarray}
340: From this, the first fixation time, $t_1$, is obtained
341: \begin{eqnarray}
342: t_1 =\underbrace{ \frac{1}{1+\sum_{k=1}^{N-1} \prod_{j=1}^{k} \gamma_j }}_{\phi_1}
343: \sum_{k=1}^{N-1} \sum_{l=1}^{k} \frac{1}{T_l^+} \prod_{j=l+1}^{k} \gamma_j.
344: \end{eqnarray}
345: Here, $\phi_1$ is the fixation probability given by Eq.~\eqref{start}.
346: Thus, the average unconditional fixation time for general $j$ is finally given by
347: \begin{eqnarray}
348: t_{j} =-\sum_{k=j+1}^N z_k = - t_1\sum_{k=j}^{N-1} \prod_{m=1}^{k} \gamma_m +\sum_{k=j}^{N-1} \sum_{l=1}^{k} \frac{1}{T_l^+} \prod_{m=l+1}^{k} \gamma_m.
349: \label{uncondavfixtime}
350: \end{eqnarray}
351: It is important to notice that the variance of these fixation times is usually high, depending on the
352: population size, the game and the intensity of selection \cite{traulsen:2007cc,dingli:2007aa}. In particular for coexistence games, where the replicator dynamics predicts a stable coexistence of $A$ and $B$, the fixation times do not only diverge with the population size and the intensity of selection, but they represent averages of a very broad distribution \cite{traulsen:2007cc}.
353:
354: \paragraph{Conditional fixation times.}
355: Given that the process reaches the absorbing state with $A$ individuals only, how long does this take
356: when starting in state $j$?
357: To calculate this time $t_A^j$, we follow Antal and Scheuring \cite{antal:2006aa}. It is convenient to start from
358: \begin{eqnarray}
359: \phi_j t^A_j = \phi_{j-1} T^-_j(t^A_{j-1}+1) +\phi_{j} (1-T^-_j-T^+_j)(t^A_{j}+1) + \phi_{j+1} T^+_j(t^A_{j+1}+1). \nonumber
360: \end{eqnarray}
361: Here, $\phi_j$ is the fixation probability of $j$ individuals of type $A$, see Eq.~\eqref{fixprob}.
362: With the abbreviation $\theta_{j}^A = \phi_j t_j^A$,
363: we can write this as
364: \begin{eqnarray}
365: \underbrace{\theta_{j+1}^A -\theta_{j}^A }_{w_{j+1}} =
366: \underbrace{\theta_{j}^A -\theta_{j-1}^A }_{w_{j}} \frac{T_j^-}{T_j^+}
367: - \frac{\phi_j}{T_j^+}.
368: \label{timeeq2}
369: \end{eqnarray}
370: Eq.~\eqref{timeeq2} has the same structure as Eq.~\eqref{timeq1}. Thus, we can use a similar iteration as above to obtain
371: \begin{eqnarray}
372: w_k & = & \theta^A_{k}-\theta^A_{k-1} = \theta^A_1 \prod_{m=1}^{k-1} \gamma_m - \sum_{l=1}^{k-1} \frac{\phi_l}{T_l^+} \prod_{m=l+1}^{k-1} \gamma_m .
373: \end{eqnarray}
374: At the lower boundary, we have $\theta^A_0=0$, because $\phi_0=0$. We also have $\theta^A_N=0$ at the upper boundary, because $t^A_N=0$.
375: Summing over $w_k$ leads to
376: $\sum_{k=j+1}^N w_k =
377: - \theta^A_{j}
378: $.
379: In particular, for $j=1$, we obtain
380: \begin{eqnarray}
381: t_1^A =
382: \sum_{k=1}^{N-1} \sum_{l=1}^{k} \frac{\phi_l}{T_l^+} \prod_{m=l+1}^{k} \gamma_m.
383: \label{t1aeq}
384: \end{eqnarray}
385: Often, this is the quantity of interest, because it corresponds to the average time it takes
386: for a single mutation to reach fixation in the population.
387: For general $j$, we have
388: \begin{eqnarray}
389: t_j^A = - t_1^A \frac{\phi_1}{\phi_j}
390: \sum_{k=j}^{N-1} \prod_{m=1}^{k} \gamma_m
391: +
392: \sum_{k=j}^{N-1} \sum_{l=1}^{k} \frac{\phi_l}{\phi_j}\frac{1}{T_l^+} \prod_{m=l+1}^{k} \gamma_m.
393: \end{eqnarray}
394: For $\phi_1=\phi_j=1$ (certain fixation of $A$), $t_j^A$ reduces to the unconditional fixation time,
395: Eq.~\eqref{uncondavfixtime}.
396:
397: For completeness, let us also calculate the average time $t_j^B$ until type $B$ reaches fixation in the population. Instead of Eq.~\eqref{timeeq2}, we now have
398: \begin{eqnarray}
399: \underbrace{\theta_{j}^B -\theta_{j-1}^B }_{v_{j}} =
400: \underbrace{\theta_{j+1}^B -\theta_{j}^B }_{v_{j+1}}
401: \frac{1}{\gamma_j}
402: + \frac{\tilde \phi_j}{T_j^-},
403: \end{eqnarray}
404: where $\tilde \phi_j=1-\phi_j$ is the fixation probability for reaching state $B$
405: and $\theta_{j}^B =\tilde \phi_j t_j^B$. Again, we have $\theta_{0}^B=\theta_{N}^B =0$.
406: Now, we start our iteration from $j=N-1$,
407: \begin{eqnarray}
408: v_{N} &=& \theta^B_{N} -\theta^B_{N-1} = -\theta^B_{N-1}\\
409: v_{N-1} &=& \theta^B_{N-1} -\theta^B_{N-2} =
410: - \theta^B_{N-1} \frac{1}{ \gamma_{N-1}} + \frac{\tilde \phi_{N-1}}{T_{N-1}^-}\\
411: & \vdots & \nonumber \\
412: v_{N-k} & = &\theta^B_{N-k} -\theta^B_{N-k-1} =
413: -\theta_{N-1}^B \prod_{m=1}^{k} \frac{1}{\gamma_{N-m}}
414: +
415: \sum_{l=1}^{k} \frac{\tilde \phi_{N-l}}{T_{N-l}^-} \prod_{m=l+1}^{k} \frac{1}{\gamma_{N-m}}
416: \nonumber.
417: \end{eqnarray}
418: Summation yields $\sum_{k=N-j}^{N-1} v_{N-k}=\theta^B_j$. From $j=N-1$, we find for the fixation time of a single $B$ mutant
419: \begin{eqnarray}
420: t_{N-1}^B = \sum_{k=1}^{N-1} \sum_{l=1}^k \frac{\tilde \phi_{N-l}}{T_{N-l}^-}\prod_{m=l+1}^{k} \frac{1}{\gamma_{N-m}}.
421: \end{eqnarray}
422: Of course, this quantity can also obtained from Eq.~\eqref{t1aeq} by symmetry arguments.
423: Finally, we obtain for the time until fixation of $B$ when
424: starting from an arbitrary number of $B$ individuals
425: \begin{eqnarray}
426: t_j^B =
427: - t_{N-1}^B \frac{\tilde \phi_{N-1}}{\tilde \phi_{j}} \sum_{k=N-j}^{N-1} \prod_{m=1}^k \frac{1}{\gamma_{N-m}}
428: +
429: \sum_{k=N-j}^{N-1} \sum_{l=1}^{k} \frac{\tilde \phi_{N-l}}{\tilde \phi_{j}} \frac{1}{T_{N-l}^-} \prod_{m=l+1}^{k} \frac{1}{\gamma_{N-m}}.
430: \end{eqnarray}
431: This formulation is valid for general birth-death processes. In addition to the
432: fixation probabilities, the two types of fixation
433: times are of particular interest
434: to characterize the evolutionary process
435: because they represent
436: global quantities that include information on all transition probabilities.
437:
438:
439: \newpage
440:
441:
442: \subsection{The Moran process and weak selection}
443:
444: As a specific example, let us consider the
445: frequency dependent Moran process
446: \cite{nowak:2004pw,taylor:2004wv}. Apart from calculating the above quantities for a specific case, our goal is to find simpler expressions for the fixation probabilities under weak selection.
447: In this case, we are close to neutral selection, which seems to be a biologically relevant limit \cite{crow:1970ck,ohta:2002aa}.
448:
449:
450: First, we need to
451: specify the payoffs of the two types $A$ and $B$.
452: The payoffs of A and B individuals, $\pi_A$ and $\pi_B$, are given by
453: \begin{eqnarray}
454: \pi_A &=& \frac{j-1}{N-1} a + \frac{N-j}{N-1} b
455: \label{eq29}
456: \\
457: \pi_B &=& \frac{j}{N-1} c + \frac{N-j-1}{N-1} d .
458: \label{eq30}
459: \end{eqnarray}
460: Here, we have excluded self interactions, i.e. in a population of $j$ individuals of type $A$, each one of them interacts with $j-1$ others of its type.
461: Fitness is assumed to be a linear combination of background fitness (which we set to 1) and the payoff,
462: \begin{eqnarray}
463: f_A &=& 1-w+w \pi_A \\
464: f_B &=& 1-w+w \pi_B .
465: \end{eqnarray}
466: The transition probabilities are
467: \begin{eqnarray}
468: T^+_j &=& \frac{j \; f_A}{j \; f_A + (N-j) f_B} \frac{N-j}{N} \\
469: T^-_j &=& \frac{(N-j) \; f_B}{j \; f_A + (N-j) f_B} \frac{j}{N} .
470: \end{eqnarray}
471: For the ratio of the transition probabilities, we have
472: \begin{equation}
473: \gamma_j = \frac{T^-_j}{T^+_j} = \frac{f_B}{f_A} = \frac{1-w+w \pi_B}{1-w+w \pi_A} .
474: \end{equation}
475:
476: Let us now consider the limit of weak selection, i.e.\ $w \ll 1$, and derive an approximation for the fixation probability $\phi_1$.
477: For weak selection, $\gamma_j$ simplifies to
478: \begin{eqnarray}
479: \gamma_j = \frac{1-w+w \pi_B}{1-w+w \pi_A}
480: \approx 1-w (\pi_A - \pi_B) .
481: \label{gammaapprox}
482: \end{eqnarray}
483: The product in Eq.~(\ref{start}) can then be simplified to
484: \begin{eqnarray}
485: \prod_{j=1}^k \gamma_j & \approx & \prod_{j=1}^k (1-w(\pi_A-\pi_B)) \approx 1-w \sum_{j=1}^k (\pi_A-\pi_B).
486: \end{eqnarray}
487: Next, we introduce a new notation for $\pi_A-\pi_B$. From Eqs.~(\ref{eq29}) and (\ref{eq30}), we find
488: \begin{equation}
489: \label{uv}
490: \pi_A-\pi_B = \underbrace{\frac{a-b-c+d}{N-1}}_u j + \underbrace{ \frac{-a+bN-dN+d}{N-1}}_v .
491: \end{equation}
492: With this we can solve the sum over the payoff difference,
493: \begin{eqnarray}
494: \sum_{j=1}^k (\pi_A-\pi_B) = \sum_{j=1}^k (u \; j +v)
495: = u \frac{(k+1)k}{2} + v k
496: = \frac{u}{2}k^2 + \left( \frac{u}{2} +v \right) k.
497: \label{end}
498: \end{eqnarray}
499: Now we have derived a simple form for $\prod_{j=1}^k \gamma_j$ under weak selection. Let us analyze the ratio of fixation probabilities:
500: \begin{eqnarray}
501: \label{thetadef}
502: \frac{\rho_B}{\rho_A} & =& {\prod_{j=1}^{N-1} \gamma_j}
503: \approx 1-w \sum_{j=1}^{N-1} \left(\pi_A -\pi_B \right) \\ \nonumber
504: & = & 1-w \left[ \frac{u}{2}(N-1) + \frac{u}{2} +v \right] (N-1) \\ \nonumber
505: & = & 1-\frac{w}{2} \underbrace{\left[ (a-b-c+d)(N-1) - a-b-c+3d+(2b-2d)N \right]}_{\Xi}.
506: \end{eqnarray}
507: For $\Xi>0$ we have $\rho_A>\rho_B$. For large populations, $N \gg 1$, we can approximate
508: \begin{eqnarray}
509: 0< \Xi \approx
510: N(a+b-c-d),
511: \end{eqnarray}
512: which is equivalent to
513: \begin{eqnarray}
514: x^{\ast} =
515: \frac{d-b}{a-b-c+d} <\frac{1}{2}
516: \end{eqnarray}
517: Hence, $\rho_A>\rho_B$ is equivalent to ${x^{\ast}} <\frac{1}{2}$. As we have discussed above, this condition indicates risk dominance and establishes a relation to fixation probabilities:
518: For weak selection, strategies with higher fixation probabilities have greater basins of attraction.
519:
520: Inserting Eq.~(\ref{end}) into Eq.~(\ref{start}), we obtain an approximation for the fixation probability of a single $A$ individual:
521: \begin{eqnarray}
522: \phi_1 &=& \frac{1}{1+ \sum_{k=1}^{N-1} \prod_{j=1}^{k} \gamma_j}
523: \approx \frac{1}{1+ \sum_{k=1}^{N-1} \left[1-w \left( \frac{u}{2} k^2 + \left( \frac{u}{2} +v \right) k \right) \right]}.
524: \end{eqnarray}
525: Using
526: $\sum_{k=1}^{N-1} k = {N (N-1)}/{2}$
527: and
528: $\sum_{k=1}^{N-1} k^2 = {N (N-1)(2N-1)}/{6}$,
529: the fixation probability becomes
530: \begin{eqnarray}
531: \phi_1
532: & \approx &\frac{1}{N- w u \frac{N (N-1)(2N-1)}{12} - w \left( \frac{u}{2} +v \right) \frac{N(N-1)}{2} } \\
533: \nonumber
534: & = &\frac{1}{N} + \frac{w}{4N} \underbrace{ \left[ (a-b-c+d) \frac{2N-1}{3} -a-b-c+3d +(2b-2d)N \right] }_{\Gamma}
535: \label{fixprobwsmoran}
536: \end{eqnarray}
537: The same fixation probability under weak selection is found for a large variety of processes \cite{imhof:2006aa,traulsen:2005hp,lessard:2007aa}. For general $i$, we have
538: \begin{eqnarray}
539: \phi_i
540: \approx
541: \frac{i}{N} + N w \frac{N-i}{N} \frac{i}{N}
542: \left(
543: \frac{a-b-c+d}{6(N-1)}(N+i) + \frac{-a+bN-dN+d}{2(N-1)}
544: \right).
545: \label{fixprobwsmorangeneral}
546: \end{eqnarray}
547:
548:
549: Let us now compare the fixation probability $\phi_1$ to the result for neutral selection, $w=0$.
550: This leads to the so called $1/3$-rule. Neutral selection means that we have no selective forces and that only randomness determines fixation.
551: In this case, we have $\phi_1=1/N$. Since we are only interested if the fixation probability is larger or smaller than $1/N$, we have to consider only the sign of $\Gamma$. If $\Gamma>0$, the fixation probability is larger than $1/N$.
552: For large $N$ this condition reduces to
553: \begin{equation}
554: \frac{a-b-c+d}{3} +b-d >0.
555: \end{equation}
556: This condition is equivalent to
557: \begin{equation}
558: x^{\ast}=
559: \frac{d-b}{a-b-c+d} < \frac{1}{3}.
560: \end{equation}
561: The 1/3-rule states that ``In a coordination game, the fixation probability of a strategy under weak selection is larger than $1/N$, if the unstable fixed point is closer than $1/3$ to the strategy to be replaced".
562: The intuitive justification of this rule goes back to the fact that during invasion, a single invader will interact on average $1/3$ with its own type and $2/3$ with the other type \cite{ohtsuki:2007aa}.
563:
564: If we increase the advantage of strategy $A$ in such a coordination game systematically (e.g. by increasing the payoff against itself), and hence shifting the mixed equilibrium $x^\ast$ to lower values, the following scenarios occur \cite{nowak:2004pw}:
565: \begin{center}
566: \begin{tabular}{l l}
567: $x^\ast>2/3$ & $A$ is disadvantageous and $B$ is advantageous \\
568: & ($\rho_A<1/N$ and $\rho_B>1/N$) \\
569: $2/3>x^\ast>1/2$ & $B$ is risk dominant, but both $A$ and $B$
570: are
571: disadvantageous \\
572: & ($\rho_A<\rho_B$, $\rho_A<1/N$ and $\rho_B < 1/N$) \\
573: $1/2>x^\ast>1/3$ & $A$ becomes risk dominant, but both $A$ and $B$ remain
574: still
575: \\
576: & disadvantageous ($\rho_A>\rho_B$, $\rho_A<1/N$ and $\rho_B < 1/N$) \\
577: $x^\ast<1/3$ & $A$ is advantageous and $B$ is disadvantageous \\
578: & ($\rho_A>1/N$ and $\rho_B<1/N$) \\
579: \end{tabular}
580: \end{center}
581:
582:
583: Interestingly, a analogous condition as the $1/3$-rule also holds for coexistence games. In this case, the stable interior fixed point has to be closer than $1/3$ to the strategy that is reached during fixation. In other words,
584: ``In a coexistence game, the fixation probability of a strategy under weak selection is larger than $1/N$, if the stable fixed point is further than $2/3$ to the strategy to be replaced".
585: However, fixation probabilities are only of limited interest here, as fixation times become very large for large populations.
586: The fixation times under weak selection are discussed in \cite{altrock:2008aa}.
587:
588: \subsection{The Fermi process}
589: \label{egfs}
590:
591: The Moran process leads to simple analytical results under weak selection
592: but no similar simplifications are possible for higher selection strengths.
593: In contrast, a pairwise comparison process with $p$ given by the Fermi function of the payoff difference (see above) admits simple analytical results for any intensity of selection.
594: The transition probabilities are
595: \begin{eqnarray}
596: T^{\pm}_j &=& \frac{j}{N} \frac{N-j}{N} \frac{1}{1+e^{\mp w(\pi_A - \pi_B)}}.
597: \end{eqnarray}
598: The analytical accessibility of this process is based on a very simple ratio of these transition probabilities,
599: \begin{equation}
600: \gamma_j = \frac{T^-_j}{T^+_j} = e^{-w (\pi_A- \pi_B)}.
601: \end{equation}
602: In the weak selection limit, $w \ll 1$, we recover Eq.~\eqref{gammaapprox}.
603: Thus, the fixation probabilities are identical to the Moran process
604: and the $1/3$ rule remains valid.
605: For arbitrary values of $w$,
606: let us
607: return to the ratio of fixation probabilities:
608: \begin{eqnarray}
609: \frac{\rho_B}{\rho_A} & =& {\prod_{j=1}^{N-1} \gamma_j}
610: = \exp \left[- w \sum_{j=1}^{N-1} \left(\pi_A -\pi_B \right) \right]
611: = \exp \left[- \frac{w}{2} {\Xi} \right].
612: \end{eqnarray}
613: Here, $\Xi$ is defined as in Eq.~\eqref{thetadef}. Again
614: we have $\rho_A>\rho_B$ for $\Theta>0$.
615: For large $N$, we find again that $\rho_A>\rho_B$ is equivalent to ${x^{\ast}} <\frac{1}{2}$. But now, the relation between the fixation probabilities and risk dominance is valid for arbitrary intensities of selection, not only for weak selection.
616:
617: The expressions for fixation probabilities simplify, because the products over $\gamma_j$ reduce to sums that can be solved exactly. One special case is determined by frequency independence of the payoff difference
618: $a-c=b-d$. This case has been termed ``equal gains from switching'', because switching from strategy $B$ to $A$ leads to the same payoff change,
619: irrespective of the opponents move \cite{nowak:1990aa}. In this special case, even the outer sum in Eq.~\eqref{fixprob} can be solved exactly for any $w$. We find
620: \begin{equation}
621: \phi_i = \frac{1-e^{-w\, v\, i} }{1-e^{-w\, v\, N} }.
622: \label{pd}
623: \end{equation}
624: This result is identical to the fixation probability of $k$ individuals with fixed relative fitness $r=e^{w v}$ \cite{crow:1970ck,ewens:2004qe}. Thus, a game with equal gains from switching has the same fixation properties as constant selection in frequency independent settings with fixed fitness values.
625: Since the Fermi process only depends on payoff differences, this is no surprise. But it also shows that
626: properties of constant selection apply not only for the Moran process under weak selection but for other processes as well.
627:
628: For general payoffs, we can approximate the outer sum in Eq.~\eqref{fixprob} over $k$ by an integral,
629: $\sum_{k=1}^i \ldots
630: \approx
631: \int
632: _1^i \ldots dk$ and arrive at \cite{traulsen:2006bb}
633: \begin{equation}
634: \phi_k= \frac{{\rm erf}\left[ Q_k\right]-
635: {\rm erf}\left[ Q_0\right]}
636: {{\rm erf}\left[Q_N \right]-
637: {\rm erf}\left[ Q_0\right]}.
638: \label{fixationerf}
639: \end{equation}
640: Here,
641: ${\rm erf}(x)=\frac{2}{\sqrt{\pi}}\int_0^x dy\, e^{-y^2}$ is the error function
642: and
643: $Q_k=\sqrt{\frac{w(N-1)}{ 2u}} \left(k u + v \right) $
644: with $u, v$ as in Eq.~(\ref{uv})
645: \cite{traulsen:2006bb}.
646: The result is valid for $ u \neq 0$
647: and in the limit of $u \to 0$, it reduces to Eq.~\eqref{pd}.
648: In the weak selection limit, $w\to 0$,
649: Eqs.\ (\ref{fixationerf}) and (\ref{pd})
650: recover the neutral selection result $\phi_k =k/N$.
651: Numerical simulations of the fixation probabilities agree very well with this approximation and hold even for small populations where the approximation of sums by integrals becomes inadequate (see Fig.~\ref{fig2}).
652:
653: The Fermi process covers all intensities of selection and leads to strong selection results that are outside the realm of the standard Moran process.
654: The closed expressions allow to derive approximations for the fixation probabilities under weak and under strong selection.
655: As for the Moran process, the time to fixation grows exponentially with $N$ for games with internal Nash equilibria, such that fixation will practically never occur.
656: Here, it also grows exponentially with the intensity of selection $w$.
657:
658:
659: \begin{figure}[t]
660: \def\capfrac{1}
661: \begin{center}
662: \includegraphics[width=0.8\textwidth]{Fig2.pdf}
663: \end{center}
664: \Caption{
665: Fixation probabilities for a coordination game with bistability (diagram on top). For neutral selection, $w=0$, the fixation probabilities are a linear function of the initial number of $A$ individuals
666: (solid line). With increasing intensity of selection $w$, the fixation probabilities of the pairwise comparison process (with $p$ given by the Fermi function) converge to a step function with a discontinuity at the unstable fixed point, $x^{\ast} = 37.5$. Symbols represent the exact expression and lines the continuous approximation Eq.~\eqref{fixationerf} (payoff matrix given in the figure, $N=50$). }
667: \label{fig2}
668: \end{figure}
669:
670:
671: \section{From finite to infinite populations (and back again)}
672:
673:
674: So far, we have introduced two descriptions for evolutionary game dynamics: The deterministic replicator dynamics for large populations and stochastic evolutionary game dynamics in finite populations. Here, we discuss how both dynamics are related to each other.
675: In the limit of large $N$, this is accomplished by performing a Kramers-Moyal expansion of the Master equation
676: \cite{gardiner:1985bv,kampen:1997xg}:
677: \begin{eqnarray}
678: P^{\tau+1}(j) - P^{\tau}(j)
679: &=&
680: \hphantom{+}
681: P^{\tau}(j-1) T^+_{j-1}+ P^{\tau} (j+1) T^- _{j+1}
682: \nonumber \\
683: & & -P^{\tau}(j) T^-_{j}
684: -P^{\tau}(j) T^+_{j},
685: \end{eqnarray}
686: where $P_j^{\tau}$ denotes the probability to be in state $j$ at time $\tau$. To consider large $N$, we introduce the notation
687: $x=j/N$, $t=\tau/N$ and the
688: probability density $\rho(x,t) = N\, P^{\tau}(j)$. For the transition probabilities, we replace $T^{\pm}_j \to T^{\pm}(x)$.
689: This
690: yields
691: \begin{eqnarray}
692: \rho\left(x,t \! + \! \frac{1}{N} \right)
693: - \rho\left(x,t\right) \! \! \!
694: &= & \! \!
695: \! \rho \! \left(x-\frac{1}{N},t\right) T^+(x-\frac{1}{N})
696: \nonumber
697: + \rho \! \left(x+\frac{1}{N},t\right) T^- (x+\frac{1}{N})
698: \nonumber \\ &-& \! \! \!
699: \rho\left(x,t\right) T^- (x)
700: -\rho\left(x,t\right) T^+ (x).
701: \label{ma2}
702: \end{eqnarray}
703: For $N \gg 1$, the probability densities and
704: the transition probabilities are expanded in a Taylor series at $x$ and $t$.
705: More specific, we have
706: \begin{eqnarray}
707: \rho\left(x ,t + \frac{1}{N} \right)
708: & \approx &
709: \rho\left(x,t\right)
710: + \frac{\partial}{\partial t}\rho\left(x,t\right) \frac{1}{N}
711: \\
712: \rho\left(x \pm \frac{1}{N},t\right)
713: &\approx &
714: \rho\left(x,t\right)
715: \pm \frac{\partial}{\partial x}\rho\left(x,t\right) \frac{1}{N}
716: + \frac{\partial^2}{\partial x^2}\rho \left(x,t\right) \frac{1}{2N^2}
717: \end{eqnarray}
718: and
719: \begin{eqnarray}
720: T^{\pm}(x \pm \frac{1}{N})
721: \approx
722: T^{\pm}(x)
723: \pm \frac{\partial}{\partial x}T^{\pm} (x) \frac{1}{N}
724: + \frac{\partial^2}{\partial x^2}T^{\pm} (x) \frac{1}{2N^2} .
725: \end{eqnarray}
726: Let us now look at the terms depending on their order in $1/N$.
727: The terms independent of $1/N$ cancel on both sides of Eq.~(\ref{ma2}).
728: The first non-vanishing term is of order $1/N$. On the left hand side, we have the term
729: $ \frac{\partial}{\partial t}\rho\left(x,t\right)$ and on the right hand side, we have
730: \begin{eqnarray}
731: \nonumber
732: & & -\rho(x,t) \frac{\partial}{\partial x}T^+(x)
733: +\rho(x,t) \frac{\partial}{\partial x}T^- (x)
734: -T^+ (x) \frac{\partial}{\partial x} \rho(x,t)
735: +T^- (x) \frac{\partial}{\partial x} \rho(x,t) \\
736: & & = - \frac{\partial}{\partial x} \left[ T^+(x) - T^-(x) \right] \rho(x,t) .
737: \end{eqnarray}
738: This term describes the average motion of the system.
739: In physics, it is called the drift term but in biology, it is referred to as selection term.
740: Next, we consider terms of the order $1/N^2$. On the right hand side, we have
741: \begin{eqnarray}
742: \nonumber
743: && \hphantom{+}\left( \frac{\partial}{\partial x} \rho(x,t) \right) \left( \frac{\partial}{\partial x} T^+(x) \right)
744: + \frac{1}{2}\rho(x,t) \frac{\partial^2}{\partial x^2} T^+(x) + \frac{1}{2} T^+(x) \frac{\partial^2}{\partial x^2}\rho(x,t)
745: \\
746: \nonumber
747: && +\left( \frac{\partial}{\partial x} \rho(x,t) \right) \left( \frac{\partial}{\partial x} T^-(x) \right)
748: + \frac{1}{2}\rho(x,t) \frac{\partial^2}{\partial x^2} T^-(x) + \frac{1}{2} T^-(x) \frac{\partial^2}{\partial x^2}\rho(x,t) \\
749: & &= \frac{1}{2} \frac{\partial^2}{\partial x^2} \left[ T^+(x) + T^-(x) \right] \rho(x,t) .
750: \end{eqnarray}
751: This second term, called diffusion in physics, leads to a widening of the probability distribution in the course of time. In biology, it is called genetic or neutral drift, which can be a source of confusion. In the following, higher order terms will be neglected. Thus, we can approximate Eq.~(\ref{ma2}) by
752: \begin{equation}
753: \frac{\partial}{\partial t}\rho\left(x,t\right) = - \frac{\partial}{\partial x} \underbrace{\left[ T^+(x) - T^-(x) \right]}_{a(x)} \rho(x,t) + \frac{1}{2} \frac{\partial^2}{\partial x^2} \underbrace{ \frac{ T^+(x) + T^-(x) }{N}}_{b^2(x)} \rho(x,t)
754: \end{equation}
755: This is the Fokker-Planck equation of the system, which describes the deterministic time evolution of a probability distribution.
756:
757: Equivalently, one can describe the process by a stochastic differential equation that generates a single trajectory. If the noise is microscopically uncorrelated, as in our case, the It{\^o} calculus has to applied \cite{gardiner:1985bv}. In this framework, the Fokker-Planck equation above corresponds to the stochastic differential equation
758: \begin{equation}
759: \dot x = a(x) + b(x) \xi,
760: \end{equation}
761: where $\xi$ is uncorrelated Gaussian noise, $a(x)$ is the drift term (selection in biology). In general, the diffusion term $b(x)$ (genetic drift in biology) depends not only on the composition of the population, but also on the payoffs. However, for many processes, the payoff dependence vanishes. In particular for weak selection, $b(x)$ is independent of the payoffs. Note that the noise is multiplicative and that the drift term vanishes at the boundaries $x=0$ and $x=1$, which is important to avoid that these boundaries are crossed. For additive noise, processes leading to $x<0$ or $x>1$ have to be excluded artificially \cite{traulsen:2004iq}
762:
763: For $N \to \infty$, we have $b(x) \to 0$ and only the term $a(x)$ determines the dynamics. This case reduces to the deterministic differential equation,
764: \begin{eqnarray}
765: \dot x = T^+(x) - T^-(x)
766: \end{eqnarray}
767: and recovers the replicator equation \cite{traulsen:2005hp} (see Fig.~\ref{fig3}). The same procedure also works for more than two strategies, although in this case, the mathematics
768: is more tedious \cite{traulsen:2006hp}. A similar procedure can also be applied to spatially extended systems, where a stochastic partial differential equation is obtained \cite{reichenbach:2007aa,reichenbach:2007bb}.
769: Note that we have only considered the limit $N \to \infty$, keeping everything else fixed. To perform a thermodynamical limit, the intensity of selection $w$ has to be scaled with $N$ \cite{chalub:2006cc}.
770:
771: The approximation of the process for large $N$ can be used to address the evolutionary dynamics in large populations under weak selection. For example, we can
772: verify the $1/3$-rule. Starting from the Fokker-Planck equation, the probability of fixation can be written in terms of the drift and diffusion coefficients as
773: \begin{equation}
774: \phi_j = \frac{S(j)}{S(N)}
775: \hspace{0.5cm}{\rm where} \hspace{0.3cm}
776: S(j)=\int_0^{j/N} \exp \left[ -2 \int_0^y \frac{ a(z)}{b^2(z)} dz \right] dy,
777: \label{fixation}
778: \end{equation}
779: see \cite{ewens:2004qe,gardiner:1985bv,traulsen:2006ab}.
780: For neutral selection, we have $a(z)=0$ and thus $\phi_j = \frac{j}{N}$.
781: In general, the comparison of the full fixation probability to neutral selection has to be done numerically.
782: In the limit of weak selection, $w\ll 1$ the $1/3$-rule is recovered when comparing the fixation probability of a small group of mutants to neutral selection \cite{traulsen:2006ab}.
783: More precisely, the $1/3$ rule is
784: obtained for $N w \ll 1$. For $N w \gg 1$, the result expected from the replicator equation is recovered: A disadvantageous
785: rare strategy will
786: never reach fixation with a significant probability.
787:
788: \begin{figure}[t]
789: \def\capfrac{1}
790: \begin{center}
791: \includegraphics[width=1.0\textwidth]{Fig3.pdf}
792: \end{center}
793: \Caption{
794: With increasing population size, the dynamics converges to the replicator equation in a game where strategy $A$ dominates
795: $B$ (diagram on top). We start
796: with $20\%$ individuals of type $A$ and
797: depict the strategy distribution after $T=N, 2N, 3N$ and $4N$ time steps
798: for a small population of size $N=100$ (left) as well as a large population with $N=10000$ (right). Selection shifts the distribution towards higher fractions of $A$ players, while the stochasticity of the evolutionary process widens the distribution. For small populations this quickly increases the variance, whereas for large populations the effect is barely visible and the distribution remains narrow.
799: Parameters: payoff matrix $a=2$, $b=5$, $c=1$, $d=3$, simulations with pairwise comparison based on the Fermi function with $w=1$, averages over $10^5$ realizations.
800: }
801: \label{fig3}
802: \end{figure}
803:
804:
805:
806:
807: \section{Applications}
808: %chh After the general remarks above, we will now turn to concrete examples for evolutionary game dynamics. Probably the most important examples are social dilemmas \cite{dawes:1980aa,hauert:2006fd,doebeli:2005aa}.
809: %chh In these system, selfish actions lead to an outcome very different from the social optimum.
810: %chh Examples include public transportation, public health systems or warning calls in biology.
811: %chh Another problem of cooperation which is not resolved yet is the preservation of global climate \cite{milinski:2006lr,Milinski:2008lr}.
812: %chh An important question is how cooperation can evolve in such systems.
813: %chh But game theory is more general. In Section \ref{rps}, we discuss an example of biodiversity based on game theoretic interactions.
814: Complementing the general theory of evolutionary game dynamics we now turn to the two most important applications in evolutionary biology and across behavioral sciences: the problem of cooperation and the maintenance of biodiversity. In game theory, cooperation refers to behavioral actions that benefit others at some cost to the actor. Thus, groups of cooperating individuals fare better than groups of non-cooperating defectors. However, each individual faces the temptation to defect and free-ride on the benefits produced by others. This generates a conflict of interest between the individuals and the group, which characterizes social dilemmas \cite{dawes:1980aa,hauert:2006fd,doebeli:2005aa}. Social dilemmas are abundant in nature. They represent a recurrent theme ranging from defense formations in musk oxen to defend their young against wolves \cite{hamilton:1971jt}, sentinel behavior in meerkats \cite{clutton-brock:1999aa}, predator inspection in fish \cite{milinski:1987ju,pitcher:1992aa}, grooming in baboons \cite{saunders:1988aa,stammbach:1982aa}, protein production in phages \cite{turner:1999hp,turner:2003hp} to microorganisms producing extra-cellular products such as enzymes in yeast \cite{greig:2004aa}, biofilms \cite{rainey:2003an} or antibiotic resistance \cite{neu:1992aa}, to name only a few prominent examples. However, social dilemmas also occurred on evolutionary scales and life could not have unfolded without the repeated incorporation of lower level units into higher levels entities. Every resolution of a social dilemma marks a major transition in evolutionary history: the formation of chromosomes out of replicating DNA, the transition from unicellular to multicellular organisms or from individuals to societies all require cooperation \cite{maynard-smith:1995bo}. In human interactions, social dilemmas are equally abundant in terms of social security, health care and pension plans but even more importantly when it comes to the preservation of natural resources from local to global scales, including drinking water, clean air, fisheries and climate \cite{hardin:1968mm,milinski:2006lr,Milinski:2008lr}.
815:
816: The viability of ecological systems is determined by the biodiversity, which includes species, habitat and genetic diversity \cite{tilman:nature06,hector:nature07,storch:07}. Species co-existence is promoted by non-hierarchical, cyclic interactions where $R$ beats $S$ beats $P$ beats $R$, just as in the children's game Rock-Scissors Paper. This occurs in bacterial strains of \emph{E. coli}, where a neutral strain is eliminated by a toxin producing strain, which is then outgrown by an immune but non-toxic strain, which is in turn outgrown by the neutral strain and so on. Further examples of cyclic dominance hierarchies include mating strategies in lizards \cite{sinervo:1996le,sinervo:2006aa} or competition for space in coral reef invertebrates \cite{jackson:1975aa} and links to the problem of cooperation if participation in social dilemmas is voluntary rather than compulsory \cite{hauert:2002te,hauert:2002in,semmann:2003he} (see Sect.~\ref{loner}).
817:
818:
819:
820: \subsection{The Prisoner's Dilemma}
821:
822: Let us begin with the Prisoner's Dilemma, which has a long tradition as a mathematical metaphor to analyze the problem of cooperation \cite{axelrod:1981yo,axelrod:1984yo}.
823: In the Prisoner's Dilemma, two players can choose between cooperation and defection. A cooperative act costs $c>0$, but leads to a benefit $b>c$ for the other player. Thus, the highest payoff, $b$, is obtained when only the partner is cooperating. In this case, the partner obtains $-c$. Mutual cooperation leads to $b-c$ and mutual defection a zero payoff. The game is characterized by the payoff matrix
824: \begin{equation}
825: \bordermatrix{
826: & C & D \cr
827: C & b-c & -c \cr
828: D & b & 0 \cr}.
829: \end{equation}
830: The Prisoner's Dilemma represents the most stringent form of a social dilemma because the
831: strategy $D$ dominates strategy $C$.
832: No matter what the opponent does, one should defect
833: since $b>b-c$ and $0>-c$.
834: Unilateral deviation from mutual defection decreases the payoff and hence mutual defection represents the only Nash equilibrium \cite{nash:1950ef,holt:2004mm}
835: but mutual cooperation corresponds to the social optimum ($b-c>0$).
836:
837: Note that the parametrization in terms of costs and benefits represents the most intuitive and mathematically convenient form of the Prisoner's Dilemma. However, it is equally important to note that this reflects a special case because
838: the sum of the diagonal elements equals the sum of the non-diagonal elements of the payoff matrix. In other words, the game is an example of ``equal gains from switching'' (see section \ref{egfs}). This property leads to a payoff difference between cooperators and defectors, $\pi_C-\pi_D=-c$, that is independent of the fraction of cooperators $x_C$.
839: In this special case the replicator dynamics reads
840: \begin{equation}
841: \dot x_C = - x_C(1-x_C) c
842: \end{equation}
843: and can be solved exactly:
844: $x_C(t) = x_C(0) \left[
845: x_C(0)+ \left(1-x_C(0) \right) e^{+c t}
846: \right]^{-1} .
847: $
848: The fraction of cooperators $x_C$ is always decreasing and converges to the only stable fixed point $x_C=0$. Cooperators are doomed and disappear.
849:
850: In finite populations and under weak selection, $w \ll 1$, we find, in agreement with Eq.~\eqref{fixprobwsmoran}, for the fixation probability of $i$ cooperators in a population of $N-i$ defectors
851: \begin{equation}
852: \phi_i = \frac{i}{N}- \frac{i}{N} \frac{N-i}{N}\left(c+ \frac{b}{N-1}\right) \frac{N w}{2}< \frac{i}{N}.
853: \end{equation}
854: Since $\phi_i< \frac{i}{N}$, cooperators are at a disadvantage compared to neutral mutants.
855: Similarly, for strong selection, we find from the Fermi process in the limit $w \to \infty$ the fixation probabilities $\phi_i = \delta_{i,N}$. In other words, cooperation cannot evolve from individual selection alone.
856:
857: The stark contrast between theoretical predictions of the Prisoner's Dilemma and the observed abundance of cooperation in nature calls for explanations. Over the last decades, a number of mechanisms capable of promoting cooperation in biological and social systems have been proposed \cite{nowak:2006pw,doebeli:2005aa}. Among related individuals, cooperators may thrive due to kin selection \cite{hamilton:1964bo} and competition among groups may promote cooperation through group selection \cite{wilson:1975sg,wilson:1994gs,fletcher:2004bv,traulsen:2006aa}. Conditional behavioral rules that strategically respond to previous encounters in repeated interactions or adjust their behavior according to an individuals' reputation in non-repeated settings can establish cooperation through direct \cite{trivers:1971hp} or indirect reciprocity \cite{nowak:1998is}. Local interactions in structured populations support cooperation in the Prisoner's Dilemma \cite{nowak:1992pw,ohtsuki:2006na} but not necessarily in other social dilemmas \cite{hauert:2004bo}. Finally, extensions of the strategy space that enable individuals to abstain from social dilemmas and render participation voluntary or options to punish non-cooperating interaction partners both support cooperation \cite{hauert:2002te,hauert:2007aa} (see Sects.~\ref{loner},~\ref{peer}).
858:
859:
860:
861: \subsection{Rock-Paper-Scissors}
862: \label{rps}
863:
864: Rock-paper-scissors games are the simplest example of cyclic dominance, where any strategy can be beaten by another one:
865: Rock crushes scissors, scissors cut paper and paper wraps rock.
866: This simple game not only entertains children (and adults \cite{rps}) but equally serves as a mathematical metaphor to investigate the dynamics and maintenance of biodiversity \cite{reichenbach:2007aa,szolnoki:2004aa,szabo:2007jt}.
867: The game can be characterized by the matrix
868: \begin{equation}
869: \label{Mmatrix}
870: \bordermatrix{
871: & R & P & S \cr
872: R & 0 & -s & +1 \cr
873: P & +1 & 0 & -s \cr
874: S & -s & +1 & 0 \cr
875: }.
876: \end{equation}
877: This parametrization assumes symmetric strategies
878: but the payoff for winning ($+1$) is not necessarily equal to the payoff for losing ($-s<0$).
879: For the standard choice $s=1$, we have a zero-sum game
880: -- one player's gain is the other ones' loss.
881:
882: Apart from the three trivial homogeneous equilibria, the replicator dynamics admits a non-trivial
883: equilibrium at $\boldsymbol{x^{\ast}} = (x_R,x_P,x_S) =(\frac{1}{3},\frac{1}{3},\frac{1}{3})$.
884: The dynamics of the system is determined by the Lyapunov function
885: \begin{equation}
886: H=-x_R x_P x_S .
887: \label{constantofmotion}
888: \end{equation}
889: If the determinant of the payoff matrix~(\ref{Mmatrix}), $d=1-s^3$, is positive, then
890: $\frac{\partial H}{\partial t} <0$ and
891: the interior fixed point ${\boldsymbol x^{\ast}}$ is
892: asymptotically stable.
893: For $d<0$, the fixed point ${\boldsymbol x^{\ast}}$ is unstable and
894: the system
895: approaches a heteroclinic cycle along the boundary of the simplex $S_3$.
896: Finally, for the zero-sum game with $s=1$, the function
897: (\ref{constantofmotion})
898: is a constant of motion, and the system infinitely oscillates
899: around ${\boldsymbol x^{\ast}}$, see Fig.~\ref{rpsfig}.
900:
901:
902: \begin{figure}[t]
903: \def\capfrac{1}
904: \begin{center}
905: \includegraphics[width=1.0\textwidth]{Fig4.pdf}
906: \end{center}
907: \Caption{
908: Depending on the determinant of the payoff matrix $d$, two qualitatively different scenarios of cyclic dominance are possible in the game of Rock-Paper-Scissors.
909: (a) If the determinant $d$ is positive, the interior fixed point is asymptotically stable and all orbits converge to this point.
910: (b) If the determinant $d$ is negative, the interior fixed point is unstable and the system
911: approaches a heteroclinic cycle along the boundary of the simplex
912: (the figure is produced with Bill Sandholm's Dynamo package \cite{sandholm:2007dy}).
913: }
914: \label{rpsfig}
915: \end{figure}
916:
917:
918: In finite populations,
919: the dynamics is stochastic but characteristic features can be derived
920: from the average drift towards the internal fixed point \cite{claussen:2008aa}.
921: For large $N$, the average drift of the Moran process computed over the entire simplex
922: is given by
923: \begin{eqnarray}
924: \langle \Delta H \rangle_N
925: \label{morandrift}
926: = \frac{1}{20N^2}
927: -\frac{1-s}{420N}\left(
928: \frac{1}{2}
929: -\frac{1}{N}
930: \right) w.
931: \end{eqnarray}
932: %at
933: For $N \to \infty$, the replicator equations lead to $\Delta H=0$ for
934: zero-sum games ($s=1$) as well as for neutral selection ($w=0$). In finite populations, we obtain $\langle \Delta H \rangle_N =\frac{1}{20N^2}>0$ in both cases instead.
935: For $s=1$, stochasticity turns the neutrally stable, closed orbits of infinite populations into an unstable system by introducing a drift towards the heteroclinic cycle.
936: Only in the limit $N \to \infty$, the neutrally stable orbits characterized by $\Delta H=0$ are recovered from the Moran process or other forms of finite population dynamics.
937: However, even in the infinite system it is not trivial to numerically determine these closed orbits of the replicator equation and the numerical integrator has to be chosen with great care \cite{hofbauer:1996mm}.
938:
939:
940:
941: For $s<1$, the sign of
942: $\langle \Delta H \rangle_N$ depends on the intensity of selection $w$, the payoff parameter $s$ as well as on the population size $N$. This leads to a critical population size
943: \begin{equation}
944: N_c = 2+\frac{42}{w(1-s)}.
945: \end{equation}
946: Since $42$ is the answer to life, the universe, and everything, this result is not surprising \cite{adams:1979aa}.
947: For $N<N_c$, the system cycles towards the boundaries and fixation is expected to be fast. For $N>N_c$,
948: the system converges towards ${\boldsymbol x^{\ast}}$ on average.
949:
950: This game is an example of a system that changes its qualitative dynamics if the population falls below a certain threshold. Such thresholds are often found in other evolutionary games in finite populations \cite{taylor:2004wv,nowak:2004pw,claussen:2007aa,traulsen:2005hp}.
951:
952:
953:
954: \newpage
955:
956: \subsection{\label{loner}Voluntary Public Goods Games}
957:
958:
959: So far, we focussed on interactions among pairs of individuals. Now, we turn to Public Goods Games \cite{kagel:1997aa} in which a group of $M$ players interact. Public Goods Games represent a generalization of pairwise Prisoner's Dilemma interactions to interaction groups of arbitrary size $M$ \cite{hauert:2003aa}.
960:
961: In typical Public Goods experiments, $M$ individuals have the opportunity to cooperate and invest a fixed amount $c$ into a common pool or to defect and invest nothing. The total investment is multiplied by a factor $r$ and distributed equally among all participants -- irrespective of whether they contributed or not. Thus, every invested unit returns $r c/M$ units to the investor (as well as to all other participants). If $r < M$ then rational players withhold their investments because they are costly -- but if all participants reason in this manner, no investments are made and the group foregoes the benefits of the public good. In contrast, had everybody cooperated, they would have been better off with a return of $(r-1)c$. Again we have a social dilemma and the Nash equilibrium is different from the social optimum. However, if $r > M$ then investments have a positive net return, and rational players will invest in the public good \cite{hauert:2006fd}.
962:
963: In infinite populations with a fraction $x$ cooperators and $y$ defectors ($x+y=1$) the average payoff of defectors is $\pi_D = (M-1)x r c/M$ and of cooperators it is $\pi_C=\pi_D-(1-r/M)c$. Thus, for $r<M$ cooperators decline and eventually disappear because $\pi_C<\pi_D$. However, for $r>M$ cooperators dominate ($\pi_C>\pi_D$) and eventually displace defectors. In this case cooperation evolves as a by-product \cite{connor:1995aa}.
964:
965: The above analysis is based on the assumption of compulsory participation in public goods interactions but what happens if participation is voluntary and risk averse individuals may choose to abstain? This situation can be modeled by introducing a third strategic type, the non-participating loners \cite{hauert:2002in,hauert:2002te}. Loners obtain a fixed payoff $\sigma$, which lies between the payoff in groups consisting solely of cooperators, $(r-1)c$, and the payoff in groups of defectors (which is zero). Interaction groups are randomly formed by sampling $M$ individuals from an infinite population with a fraction $x$ cooperators, $y$ defectors and $z$ loners. Thus, the effective interaction group size $S$ of the Public Goods Game decreases with increasing abundance of loners. If only a single individual participates in the Public Goods interaction, the game does not take place and the individual is forced to act as a loner. The average payoffs of defectors, cooperators and loners are given by \cite{hauert:2002in}:
966: \begin{eqnarray}
967: \pi_D & = &
968: \sigma z^{M-1} + r \; c \frac{x}{1-z} \left(1-\frac{1-z^M}{M(1-z)} \right) \nonumber\\
969: \pi_C & = & \pi_D
970: -\left(1+(r-1)z^{N-1}-\frac rM \frac{1-z^M}{1-z}\right)c \nonumber\\
971: \pi_L & = & \sigma
972: \end{eqnarray}
973: Note that with few loners, $z\to 0$, defectors dominate cooperators, $\pi_D>\pi_C$, but for $z\to 1$, cooperators outperform defectors, $\pi_C>\pi_D$. This generates a Rock-Scissors-Paper type cyclic dominance among the three strategies: if cooperators abound it pays to defect and if defection is common it is best to abstain but this reduces the effective interaction group size $S$ until eventually $r>S$ holds and cooperators thrive, which in turn increases $S$, restores the social dilemma and the cycle continues, see Fig.~\ref{lonerfig}b. By definition $S\geq 2$ must hold because a single participant ($S=1$) is unable to sustain the public good. Therefore, the above reasoning requires $r>2$. In this case, a heteroclinic orbit along the boundary of the simplex $S_3$ reflects the cyclic dominance of the three strategies and in the interior a neutrally stable fixed point $\bf Q$ exists, which is surrounded by closed orbits such that the system exhibits stable periodic oscillations of the strategy frequencies, see Fig.~\ref{lonerfig}b. For $r\leq 2$, the heteroclinic orbit still exists but the interior is filled with homoclinic orbits of the state with all loners, $z=1$. Therefore, only brief intermittent bursts of cooperation are observed before the system returns to the loner state, see Fig.~\ref{lonerfig}a.
974:
975:
976: \begin{figure}[t]
977: \def\capfrac{1}
978: \begin{center}
979: \includegraphics[width=1.0\textwidth]{Fig5.pdf}
980: \end{center}
981: \Caption{
982: Dynamics in voluntary Public Goods Games in which cooperators are dominated by defectors, defectors by loners and loners by cooperators.
983: (a) For $r \leq 2$, only brief intermittent bursts of cooperation are observed before the
984: system settles again in the loner state.
985: (b) For $r>2$, a neutrally stable fixed point $\bf Q$ appears, which is surrounded by closed orbits. The fraction of cooperators performs stable oscillations (parameters: (a) $c=1$, $r=1.8$ and $\sigma=0.5$. (b) $c=1$, $r=3$ and $\sigma=1$).
986: }
987: \label{lonerfig}
988: \end{figure}
989:
990:
991:
992: In order to allow for a compact analytical description of the stochastic dynamics in finite populations, we assume small mutation rates $\mu$
993: such that a strategy reaches fixation or extinction much faster than mutants are produced \cite{imhof:2005oz,fudenberg:2006fu,hauert:2007aa}.
994: This is fulfilled if $\mu$ is much smaller than the inverse of the squared population size. Thus, the system is usually homogeneous and only occasionally switches from one homogeneous state to another. The dynamics is essentially reduced to a stochastic process along the boundaries of the discretized simplex $S_3$ and
995: we can approximate the system by an embedded Markov chain on the pure states of the system, $C$, $D$ and $L$.
996: The average time the system spends in each state depends on the updating process, the intensity of selection as well as on the game parameters.
997: For simplicity, let us consider imitation dynamics (c.f. Eq.~(\ref{fermieq}) in the limit of strong selection, $w\to\infty$).
998: In this case,
999: a single defector takes over a cooperator population with probability $1$.
1000: Similarly, the probability that a single loner takes over a population of defectors is also $1$. Finally, a loner population is taken over by cooperators with probability $1/2$. This is because the first cooperator is neutral (no Public Goods games take place) and disappears with the same probability as it gets imitated. However, as soon as there are two cooperators, they have an advantage and eliminate the loners with certainty. This leads to the transition matrix among the three homogeneous states:
1001: \begin{equation}
1002: \label{cdltrans}
1003: \bordermatrix{
1004: & C & D & L \cr
1005: C & \frac{1}{2} & 0 & \frac{1}{4} \cr
1006: D & \frac{1}{2} & \frac{1}{2} & 0 \cr
1007: L & 0 & \frac{1}{2} & \frac{3}{4} \cr}.
1008: \end{equation}
1009: Note that the system stays e.g. in state $C$ with probability $1/2$ because the probability that the mutant is an unsuccessful loner is $1/2$. Under imitation dynamics the transition matrix is parameter independent as long as the cyclic dominance applies (which only requires $r<M$ and $0<\sigma<r-1$).
1010: The stationary distribution ${\boldsymbol P}$ is given by the eigenvector corresponding to the eigenvalue $1$ of this stochastic matrix, ${\boldsymbol P}=(P_C,P_D,P_L)=(\frac{1}{4},\frac{1}{4},\frac{1}{2})$.
1011: Thus, the system spends 50\% of the time in the loner state and 25\% in the cooperator and and defector states, respectively. In compulsory interactions, i.e. in the absence of loners,
1012: the system would spend essentially all the time in the defector state. Interactive Java simulations of this system can be found online \cite{vlabs:2008}.
1013:
1014:
1015: In summary, voluntary participation in Public Goods Games provides an escape hatch out of states of mutual defection. This maintains cooperation, but fails to stabilize it.
1016: The cyclic dynamics of voluntary Public Goods Games has been confirmed in behavioral experiments \cite{semmann:2003he}.
1017: In the next section, we demonstrate that the time spent in cooperative states can be vastly extended by introducing opportunities to punish defectors.
1018:
1019:
1020: \subsection{\label{peer}Punishment}
1021:
1022: Punishment is ubiquitous in nature ranging from bacterial colonies to human societies \cite{clutton-brock:nature95,fehr:2002bv,foster:2000le,gurerk:2006jb,sigmund:2007oz,dreber:2008lr}. In game theoretical terms, punishment is defined as behavioral actions that impose a fine $\beta$ on a co-player at some cost $\gamma<\beta$ to the actor. Punishment enables cooperators to retaliate against non-cooperating defectors and therefore seems capable of stabilizing cooperation. However, this raises a second-order social dilemma because non-punishing cooperators outperform those that do punish. Thus, in an evolving population, mild cooperators undermine the punishers' efforts and pave the way for the successful invasion of defectors. Moreover, if punishers are rare, they suffer tremendous costs from punishing left and right, while inflicting little harm on the defecting members of the population and hence it remains unclear how punishment behavior could have gained a foothold in a population. Colman aptly summarizes this by stating: ``we seem to have replaced the problem of explaining cooperation with that of explaining altruistic punishment'' \cite{colman:2006aa}.
1023:
1024: In the previous section we demonstrated that voluntary participation in Public Goods Games promotes cooperation without being able to stabilize it. Here we extend this setting by introducing a fourth strategic type, the punishers. Punishers are individuals that cooperate but, in addition, punish defectors. The combination of volunteering an punishment was originally proposed by Fowler \cite{fowler:2005aa}, but it turns out that the replicator dynamics in infinite populations is unsuitable to draw clear-cut conclusions because the system is bi-stable (i.e. the evolutionary end state depends on the initial configuration) and also structurally unstable (i.e. the evolutionary dynamics may undergo significant changes upon introducing stochastic components) \cite{brandt:2006aa}.
1025:
1026: In the following, we consider the stochastic dynamics of volunteering and punishment in Public Goods Games in finite populations \cite{hauert:2007aa,hauert:2008bb}. As in the case without punishment, we take the limit of small mutation rates and strong selection. Again, this implies that the population is homogeneous most of the time, i.e. in states $C, D, L$ or $P$. An occasional mutant will have taken over the population or disappeared before the next mutation arises. Therefore, the dynamics is determined by a Markov chain based on the different transition probabilities. In the limit of strong selection, the derivation of these transition probabilities is particularly simple:
1027: (i) in the cooperator state $C$, a single defector is advantageous and takes over the population with probability $1$.
1028: Loners are disadvantageous and cannot invade.
1029: Punishers can invade through neutral drift with probability $\frac{1}{N}$.
1030: (ii) in the defector state $D$, a mutant cooperator or punisher is disadvantageous and disappears.
1031: Loners are advantageous and take over with probability $1$.
1032: (iii) in the loner state $L$, the dynamics is neutral if a mutant of any type arises. With probability $1/2$ another individual adopts the mutant strategy (or the mutant disappears with the same probability). A pair of mutant cooperators or punishers is advantageous and takes over but a pair of defectors is disadvantageous and at least one disappears. (iv) in the punisher state $P$, both defectors and loners are disadvantageous. The former because they are punished and the latter because they do not take advantage of the common good. However, cooperators obtain the same payoff as punishers and can take over through neutral drift with probability $\frac{1}{N}$.
1033: This yields the transition matrix
1034: \begin{equation}
1035: \bordermatrix{
1036: & C & D & L & P \cr
1037: C & \frac{2}{3}-\frac{1}{3N} & 0 & \frac{1}{6} & \frac{1}{3N} \cr
1038: D & \frac{1}{3} & \frac{2}{3} & 0 & 0 \cr
1039: L & 0 & \frac{1}{3} & \frac{2}{3}& 0 \cr
1040: P & \frac{1}{3N} & 0 & \frac{1}{6} & 1-\frac{1}{3N} \cr} ,
1041: \end{equation}
1042: which is again independent of the interaction parameters (c.f. Eq.~(\ref{cdltrans})) and only depends on the population size $N$ because of the neutral transitions between cooperators and punishers.
1043: The stationary distribution becomes
1044: \begin{equation}
1045: {\boldsymbol P}=(P_C,P_D,P_L,P_P)=
1046: \left(
1047: \frac{2}{8+N},\frac{2}{8+N},\frac{2}{8+N},\frac{2+N}{8+N}
1048: \right).
1049: \end{equation}
1050: Thus, for large $N$, the system spends almost all the time in the punisher state. The reason is that the transition leading away from the punisher state is neutral and thus very slow compared to all other transitions. The punisher state is the only pure Nash equilibrium, but it is not a strict Nash equilibrium, as cooperators are equally well off. Despite the vanishing time the system spends in the loners' state, voluntary participation plays a pivotal role for cooperation and punishment because it provides recurrent opportunities for establishing social norms based on punishing non-cooperating defectors, see Fig.~\ref{smarties}.
1051:
1052: \begin{figure}[t]
1053: \def\capfrac{1}
1054: \begin{center}
1055: \includegraphics[width=1.0\textwidth]{Fig6.pdf}
1056: \end{center}
1057: \Caption{
1058: For small mutation rates and strong selection, the stochastic dynamics of finite populations can be approximated by an embedded Markov chain on the pure states of the system.
1059: (a) For the voluntary Public Goods Game, a cyclic dominance between cooperators, defectors and loners emerges. We find that the system spends 50\% in the Loner state and 25 \% in the cooperator and defector states, respectively. This result is independent of all parameters.
1060: (b) In voluntary Public Goods Games with punishment, punishers can invade the loner state. The system then spends a long time in the punisher state because only cooperators can invade and only through neutral drift. Once the system is in the cooperator state it is prone to invasion by defectors and may undergo several cooperator-defector-loner-cycles before punishment is re-established. In the long run, punishers dominate the population most of the time.
1061: (c) In compulsory Public Goods Games with punishment, the ultimate outcome is defection. Even when starting in the punisher state, cooperators can invade by neutral drift and
1062: once defectors take over, the system has reached its evolutionary end state
1063: (population size $N=92$).}
1064: \label{smarties}
1065: \end{figure}
1066:
1067: In contrast, in
1068: compulsory Public Goods Games, i.e.\ in the absence of loners,
1069: the cyclic dominance of $C$, $D$, and $L$ is missing and once cooperation breaks down, it cannot get re-established, see Fig.~\ref{smarties}.
1070: Interestingly, punishment emerges only if the participation in Public Goods Games is voluntary. This conclusion nicely relates to recent experiments, where individuals can choose whether to join a Public Goods Game with or without punishment, they voluntarily commit themselves to the sanctioning rules \cite{gurerk:2006jb}.
1071: For interactive simulations of volunteering and punishment, see \cite{vlabs:2008}.
1072:
1073: This approach to punishment, which is most common in behavioral economics, raises moral concerns because enforcing cooperation through peer-punishment means that individuals take the law into their own hands, but mob law is clearly socially unacceptable.
1074:
1075: \section{Concluding remarks}
1076: This review mainly focusses on the particularly insightful class of $2 \times 2$ games.
1077: We have analyzed the stochastic dynamics of evolutionary games in
1078: finite populations, as it is, for example, described by the Moran process.
1079: The connection to the traditional description of evolutionary games by the
1080: deterministic replicator equation is established
1081: through approximations in the limit of large populations.
1082:
1083: As applications and extensions of the theoretical framework, we provide brief excursions into interactions with more strategic options, as in the case of Rock-Scissors-Paper games, which are relevant in the context of biodiversity, as well as to interactions among larger groups of individuals to address the problem of cooperation in the Prisoner's Dilemma and in Public Goods Games with voluntary participation and/or punishment opportunities.
1084: Other important games that have not been covered here include the
1085: Snowdrift Game \cite{doebeli:2004bo,hauert:2004bo}, where cooperation is not
1086: strictly dominated by defection, and the Minority Game, which turned into a paradigm
1087: for simplified market models \cite{challet:1997uw,challet:2004bo,coolen:2005bo}.
1088:
1089: Further important directions of the field include
1090: spatial games, which have recently been summarized in
1091: an excellent review by Szab{\'o} and F{\'a}th \cite{Szabo:2007aa}, as well as recent advances in
1092: ecological games with variable population densities \cite{hauert:2006ha,hauert:tpb08}
1093: and for
1094: games with continuous strategy spaces \cite{killingback:procb99,doebeli:2004bo}
1095: based on
1096: adaptive dynamics \cite{dieckmann:jmb96,geritz:evolecol98,metz:96}.
1097:
1098: Over the past few years, the tremendous progress of our understanding of evolutionary dynamics can be largely attributed to numerous insights gained from stochastic dynamics in finite populations as well as from considering the analytically accessible limits of weak selection and rare mutations. The often counter intuitive results
1099: are inaccessible from traditional approaches based on replicator equations.
1100:
1101: Stochastic evolutionary game dynamics is more difficult to handle analytically as compared to deterministic approaches such as the replicator dynamics. Nonetheless, it is a very powerful tool, because it
1102: implements a natural source of noise that actually renders the results more robust. For example, in bistable systems it allows to calculate transition rates between the different
1103: states
1104: rather than concluding that the dynamics depends on the initial conditions. Most importantly, any real population is subject to noise and incorporating
1105: such essential features into game theoretical scenarios significantly improves their relevance in modeling the real world.
1106:
1107:
1108:
1109: \subsection*{Acknowledgements}
1110: We thank W.H. Sandholm for providing the software for Fig.~4.
1111: A.T.\ acknowledges support by the ``Deutsche Akademie der Naturforscher Leopoldina'' (Grant No.\ BMBF-LPD 9901/8-134) and the Emmy-Noether program of the DFG.
1112: C.H.\ is supported by the John Templeton Foundation.
1113:
1114: \setlength{\bibindent}{6mm} % room for up to 3 digits
1115: \renewcommand{\bibname}{References}
1116: \begin{thebibliography}{100}
1117:
1118: \bibitem{adams:1979aa}
1119: D.~Adams.
1120: \newblock {\em The Hitchhiker's Guide to the Galaxy}.
1121: \newblock Pan Books, 1979.
1122:
1123: \bibitem{altrock:2008aa}
1124: P.M.~Altrock and A.~Traulsen.
1125: \newblock Fixation times in evolutionary games under weak selection.
1126: \newblock{\em New Journal of Physics}, in press, 2008.
1127:
1128: \bibitem{antal:2006aa}
1129: T.~Antal and I.~Scheuring.
1130: \newblock Fixation of strategies for an evolutionary game in finite
1131: populations.
1132: \newblock {\em Bull. Math. Biol.}, 68:1923--1944, 2006.
1133:
1134: \bibitem{axelrod:1981yo}
1135: R.~Axelrod and W.~D. Hamilton.
1136: \newblock The evolution of cooperation.
1137: \newblock {\em Science}, 211:1390--1396, 1981.
1138:
1139: \bibitem{axelrod:1984yo}
1140: R.~Axelrod.
1141: \newblock {\em The Evolution of Cooperation}.
1142: \newblock Basic Books, New York, 1984.
1143:
1144: \bibitem{brandt:2006aa}
1145: H.~Brandt, C.~Hauert, and K.~Sigmund.
1146: \newblock Punishing and abstaining for public goods.
1147: \newblock {\em Proc. Natl. Acad. Sci. USA}, 103:495--497, 2006.
1148:
1149: \bibitem{challet:2004bo}
1150: D.~Challet, M.~Marsili, and Y.-C. Zhang.
1151: \newblock {\em Minority games: Interacting agents in financial markets}.
1152: \newblock Oxford University Press, Oxford, 2004.
1153:
1154: \bibitem{challet:1997uw}
1155: D.~Challet and Y.-C. Zhang.
1156: \newblock Emergence of cooperation and organization in an evolutionary game.
1157: \newblock {\em Physica A}, 246:407--418, 1997.
1158:
1159: \bibitem{chalub:2006cc}
1160: F.~A. C.~C. Chalub and M.~O. Souza.
1161: \newblock The continuous limit of the moran process and the diffusion of mutant
1162: genes in infinite populations.
1163: \newblock {\em math.AP/0602530}, 2006.
1164:
1165: \bibitem{claussen:2007aa}
1166: J.~C. Claussen.
1167: \newblock Drift reversal in asymmetric coevolutionary conflicts: influence of
1168: microscopic processes and population size.
1169: \newblock {\em European Physical Journal B}, 60:391--399, 2007.
1170:
1171: \bibitem{claussen:2008aa}
1172: J.~C. Claussen and A.~Traulsen.
1173: \newblock Cyclic dominance and biodiversity in well-mixed populations.
1174: \newblock {\em Phys. Rev. Lett.}, 100:058104, 2008.
1175:
1176: \bibitem{clutton-brock:nature95}
1177: T.~H. Clutton-Brock and G.~A. Parker.
1178: \newblock Punishment in animal societies.
1179: \newblock {\em Nature}, 373:209--216, 1995.
1180:
1181: \bibitem{clutton-brock:1999aa}
1182: T.~H. Clutton-Brock, P.~N. M.~Brotherton M.~J.~O'Riain, D.~Gaynor, R.~Kansky,
1183: A.~S. Griffin, and M.~Manser.
1184: \newblock Selfish sentinels in cooperative mammals.
1185: \newblock {\em Science}, 284:1640--1644, 1999.
1186:
1187: \bibitem{colman:2006aa}
1188: A.~M. Colman.
1189: \newblock The puzzle of cooperation.
1190: \newblock {\em Nature}, 440:744--745, 2006.
1191:
1192: \bibitem{connor:1995aa}
1193: R.~C. Connor.
1194: \newblock Altruism among non-relatives: alternatives to the '{P}risoner's
1195: {D}ilemma'`.
1196: \newblock {\em TREE}, 10(2):84--86, 1995.
1197:
1198: \bibitem{coolen:2005bo}
1199: A.~C.~C. Coolen.
1200: \newblock {\em The mathematical theory of minority games: Statstical mechanics
1201: of interacting agents}.
1202: \newblock Oxford University Press, Oxford, 2005.
1203:
1204: \bibitem{crow:1970ck}
1205: J.~F. Crow and M.~Kimura.
1206: \newblock {\em An introduction to population genetics theory}.
1207: \newblock Harper and Row, New York, NY, 1970.
1208:
1209: \bibitem{dawes:1980aa}
1210: R.~M. Dawes.
1211: \newblock Social dilemmas.
1212: \newblock {\em Ann. Rev. Psych.}, 31:169--193, 1980.
1213:
1214: \bibitem{dieckmann:jmb96}
1215: U.~Dieckmann and R.~Law.
1216: \newblock The dynamical theory of coevolution: a derivation from stochastic
1217: ecological processes.
1218: \newblock {\em J. Math. Biol.}, 34:579--612, 1996.
1219:
1220: \bibitem{dingli:2007aa}
1221: D.~Dingli, A.~Traulsen, and J.~M. Pacheco.
1222: \newblock Stochastic dynamics of hematopoietic tumor stem cells.
1223: \newblock {\em Cell Cycle}, 6:e2--e6, 2007.
1224:
1225: \bibitem{doebeli:2005aa}
1226: M.~Doebeli and C.~Hauert.
1227: \newblock Models of cooperation based on the prisoner's dilemma and the
1228: snowdrift game.
1229: \newblock {\em Ecology Letters}, 8:748--766, 2005.
1230:
1231: \bibitem{doebeli:2004bo}
1232: M.~Doebeli, C.~Hauert, and T.~Killingback.
1233: \newblock The evolutionary origin of cooperators and defectors.
1234: \newblock {\em Science}, 306:859--62, 2004.
1235:
1236: \bibitem{dreber:2008lr}
1237: A.~Dreber, D.~G. Rand, D.~Fudenberg, and M.~A. Nowak.
1238: \newblock Winners don't punish.
1239: \newblock {\em Nature}, 452:348--351, 2008.
1240:
1241: \bibitem{ewens:2004qe}
1242: W.~J. Ewens.
1243: \newblock {\em Mathematical Population Genetics}.
1244: \newblock Springer, New York, 2004.
1245:
1246: \bibitem{fehr:2002bv}
1247: E.~Fehr and S.~G\"{a}chter.
1248: \newblock Altruistic punishment in humans.
1249: \newblock {\em Nature}, 415:137--140, 2002.
1250:
1251: \bibitem{fletcher:2004bv}
1252: J.~A. Fletcher and M.~Zwick.
1253: \newblock Strong altruism can evolve in randomly formed groups.
1254: \newblock {\em Jour. Theor. Biol.}, 228:303--313, 2004.
1255:
1256: \bibitem{foster:2000le}
1257: K.~R. Foster and F.~L.~W. Ratnieks.
1258: \newblock Facultative worker policing in a wasp.
1259: \newblock {\em Nature}, 407:692--693, 2000.
1260:
1261: \bibitem{fowler:2005aa}
1262: J.~H. Fowler.
1263: \newblock Altruistic punishment and the origin of cooperation.
1264: \newblock {\em Proc. Natl. Acad. Sci. USA}, 102:7047--7049, 2005.
1265:
1266: \bibitem{fudenberg:2006fu}
1267: D.~Fudenberg, M.~A. Nowak, C.~Taylor, and L.A. Imhof.
1268: \newblock Evolutionary game dynamics in finite populations with strong
1269: selection and weak mutation.
1270: \newblock {\em Theor. Pop. Biol.}, 70:352--363, 2006.
1271:
1272: \bibitem{gardiner:1985bv}
1273: C.~W. Gardiner.
1274: \newblock {\em Handbook of Stochastic Methods}.
1275: \newblock Springer, Berlin, 2nd edition, 1985.
1276:
1277: \bibitem{geritz:evolecol98}
1278: S.~A.~H. Geritz, \'E. Kisdi, G.~Mesz\'{e}na, and J.~A.~J. Metz.
1279: \newblock Evolutionarily singular strategies and the adaptive growth and
1280: branching of the evolutionary tree.
1281: \newblock {\em Evol. Ecol.}, 12:35--57, 1998.
1282:
1283: \bibitem{gigerenzer:2002aa}
1284: G.~Gigerenzer and R.~Selten.
1285: \newblock {\em Bounded rationality}.
1286: \newblock MIT Press, 2002.
1287:
1288: \bibitem{greig:2004aa}
1289: D. Greig and M. Travisano.
1290: \newblock The {P}risoner's {D}ilemma and polymorphism in yeast {SUC} genes.
1291: \newblock {\em Biol. Lett.}, 271:S25 -- S26, 2004.
1292:
1293: \bibitem{gurerk:2006jb}
1294: {\"O}. G{\"u}rerk, B. Irlenbusch, and B. Rockenbach.
1295: \newblock The competitive advantage of sanctioning institutions.
1296: \newblock {\em Science}, 312:108--111, 2006.
1297:
1298: \bibitem{hamilton:1964bo}
1299: W.~D. Hamilton.
1300: \newblock The genetical evolution of social behavior {I} and {II}.
1301: \newblock {\em J. Theor. Biol.}, 7:1--16 + 17--52, 1964.
1302:
1303: \bibitem{hamilton:1971jt}
1304: W.~D. Hamilton.
1305: \newblock The geometry of the selfish herd.
1306: \newblock {\em J. theor. Biol.}, 31:295--311, 1971.
1307:
1308: \bibitem{hardin:1968mm}
1309: G.~Hardin.
1310: \newblock The tragedy of the commons.
1311: \newblock {\em Science}, 162:1243--1248, 1968.
1312:
1313: \bibitem{vlabs:2008}
1314: C.~Hauert.
1315: \newblock Virtual labs.
1316: \newblock {\em http://www.univie.ac.at/virtuallabs/}, 2008.
1317:
1318: \bibitem{hauert:2002in}
1319: C.~Hauert, S.~De~Monte, J.~Hofbauer, and K.~Sigmund.
1320: \newblock Replicator dynamics for optional public good games.
1321: \newblock {\em J. Theor. Biol.}, 218:187--94, 2002.
1322:
1323: \bibitem{hauert:2002te}
1324: C.~Hauert, S.~De~Monte, J.~Hofbauer, and K.~Sigmund.
1325: \newblock Volunteering as red queen mechanism for cooperation in public goods
1326: games.
1327: \newblock {\em Science}, 296:1129--1132, 2002.
1328:
1329: \bibitem{hauert:2004bo}
1330: C.~Hauert and M.~Doebeli.
1331: \newblock Spatial structure often inhibits the evolution of cooperation in the
1332: snowdrift game.
1333: \newblock {\em Nature}, 428:643--646, 2004.
1334:
1335: \bibitem{hauert:2006ha}
1336: C.~Hauert, M.~Holmes, and M.~Doebeli.
1337: \newblock Evolutionary games and population dynamics:maintenance of cooperation
1338: in public goods games.
1339: \newblock {\em Proc. Roy. Soc. Lond. B}, 273:2565--2570, 2006.
1340:
1341: \bibitem{hauert:2006fd}
1342: C.~Hauert, F.~Michor, M.~A. Nowak, and M.~Doebeli.
1343: \newblock Synergy and discounting of cooperation in social dilemmas.
1344: \newblock {\em J. Theor. Biol.}, 239:195--202, 2006.
1345:
1346: \bibitem{hauert:2003aa}
1347: C.~Hauert and G.~Szab{\'o}.
1348: \newblock Prisoner's dilemma and public goods games in different geometries:
1349: compulsory versus voluntary interactions.
1350: \newblock {\em Complexity}, 8:31--38, 2003.
1351:
1352: \bibitem{hauert:2007aa}
1353: C.~Hauert, A.~Traulsen, H.~Brandt, M.~A. Nowak, and K.~Sigmund.
1354: \newblock Via freedom to coercion: the emergence of costly punishment.
1355: \newblock {\em Science}, 316:1905--1907, 2007.
1356:
1357: \bibitem{hauert:2008bb}
1358: C.~Hauert, A.~Traulsen, H.~Brandt, M.~A. Nowak, and K.~Sigmund.
1359: \newblock Public goods with punishment and abstaining in finite and infinite
1360: populations.
1361: \newblock {\em Biological Theory}, in press, 2008.
1362:
1363: \bibitem{hauert:tpb08}
1364: C. Hauert, J.~Y. Wakano, and M. Doebeli.
1365: \newblock Ecological public goods games: cooperation and bifurcation.
1366: \newblock {\em Theor. Pop. Biol.}, 73:257--263, 2008.
1367:
1368: \bibitem{hector:nature07}
1369: A. Hector and R. Bagchi.
1370: \newblock Biodiversity and ecosystem multifunctionality.
1371: \newblock {\em Nature}, 448:188--191, 2007.
1372:
1373: \bibitem{hofbauer:1996mm}
1374: J.~Hofbauer.
1375: \newblock Evolutionary dynamics for bimatrix games: A {H}amiltonian system?
1376: \newblock {\em J. Math. Biol.}, 34:675--688, 1996.
1377:
1378: \bibitem{hofbauer:1998mm}
1379: J.~Hofbauer and K.~Sigmund.
1380: \newblock {\em Evolutionary Games and Population Dynamics}.
1381: \newblock Cambridge University Press, Cambridge, 1998.
1382:
1383: \bibitem{holt:2004mm}
1384: C.~A. Holt and A.~E. Roth.
1385: \newblock The {N}ash equilibrium: A perspective.
1386: \newblock {\em Proc. Natl. Acad. Sci. USA}, 101:3999--4002, 2004.
1387:
1388: \bibitem{rps}
1389: http://www.worldrps.com/.
1390:
1391: \bibitem{imhof:2005oz}
1392: L.~A. Imhof, D.~Fudenberg, and M.~A. Nowak.
1393: \newblock Evolutionary cycles of cooperation and defection.
1394: \newblock {\em Proc. Natl. Acad. Sci. USA}, 102:10797--10800, 2005.
1395:
1396: \bibitem{imhof:2006aa}
1397: L.~A. Imhof and M.~A. Nowak.
1398: \newblock Evolutionary game dynamics in a {W}right {F}isher process.
1399: \newblock {\em J. Math. Biol.}, 52:667--681, 2006.
1400:
1401: \bibitem{jackson:1975aa}
1402: J.~B.~C. Jackson and L.~Buss.
1403: \newblock Allelopathy and spatial competition among coral reef invertebrates.
1404: \newblock {\em Proc. Natl. Acad. Sci. USA}, 72:5160--5163, 1975.
1405:
1406: \bibitem{kagel:1997aa}
1407: J.~H. Kagel and A.~E. Roth, editors.
1408: \newblock {\em The Handbook of Experimental Economics}.
1409: \newblock Princeton Univ. Press, 1997.
1410:
1411: \bibitem{kampen:1997xg}
1412: N.~G.~van Kampen.
1413: \newblock {\em Stochastic Processes in Physics and Chemistry}.
1414: \newblock Elsevier, Amsterdam, 2 edition, 1997.
1415:
1416: \bibitem{kerr:2002xg}
1417: B.~Kerr, M.~A. Riley, M.~W. Feldman, and B.~J.~M. Bohannan.
1418: \newblock Local dispersal promotes biodiversity in a real-life game of
1419: rock-paper-scissors.
1420: \newblock {\em Nature}, 418:171--174, 2002.
1421:
1422: \bibitem{killingback:procb99}
1423: T. Killingback, M. Doebeli, and N. Knowlton.
1424: \newblock Variable investment, the continuous prisoner's dilemma, and the
1425: origin of cooperation.
1426: \newblock {\em Proc. R. Soc. Lond. B}, 266:1723--1728, 1999.
1427:
1428: \bibitem{lessard:2007aa}
1429: S.~Lessard and V.~Ladret.
1430: \newblock The probability of fixation of a single mutant in an exchangeable
1431: selection model.
1432: \newblock {\em J. Math. Biol.}, 54:721--744, 2007.
1433:
1434: \bibitem{maynard-smith:1982to}
1435: J.~Maynard~Smith.
1436: \newblock {\em Evolution and the Theory of Games}.
1437: \newblock Cambridge University Press, Cambridge, 1982.
1438:
1439: \bibitem{maynard-smith:1973to}
1440: J.~Maynard~Smith and G.~R. Price.
1441: \newblock The logic of animal conflict.
1442: \newblock {\em Nature}, 246:15--18, 1973.
1443:
1444: \bibitem{maynard-smith:1995bo}
1445: J.~Maynard~Smith and Szathm{\'a}ry.
1446: \newblock {\em The major transitions in evolution}.
1447: \newblock W. H. Freeman, Oxford, 1995.
1448:
1449: \bibitem{metz:96}
1450: J.~A.~J. Metz, S.~A.~H. Geritz, G.~Meszena, F.~J.~A. Jacobs, and J.~S. van
1451: Heerwaarden.
1452: \newblock Adaptive dynamics: a geometrical study of the consequences of nearly
1453: faithful replication.
1454: \newblock In S.~J. van Strien and S.~M. Verduyn~Lunel, editors, {\em Stochastic
1455: and Spatial Structures of Dynamical Systems}, pages 183--231. North Holland,
1456: Amsterdam, 1996.
1457:
1458: \bibitem{milinski:1987ju}
1459: M.~Milinski.
1460: \newblock Tit for tat in sticklebacks and the evolution of cooperation.
1461: \newblock {\em Nature}, 325:433--435, 1987.
1462:
1463: \bibitem{milinski:2006lr}
1464: M.~Milinski, D.~Semmann, H.-J. Krambeck, and J.~Marotzke.
1465: \newblock Stabilizing the earth's climate is not a losing game: Supporting
1466: evidence from public goods experiments.
1467: \newblock {\em Proc. Natl. Acad. Sci. USA}, 103:3994--3998, 2006.
1468:
1469: \bibitem{Milinski:2008lr}
1470: M.~Milinski, R.~D. Sommerfeld, H.-J. Krambeck, F.~A. Reed, and J.~Marotzke.
1471: \newblock The collective-risk social dilemma and the prevention of simulated
1472: dangerous climate change.
1473: \newblock {\em Proc. Natl. Acad. Sci. USA}, 105:2291--2294, 2008.
1474:
1475: \bibitem{moran:1962ef}
1476: P.~A.~P. Moran.
1477: \newblock {\em The statistical processes of evolutionary theory}.
1478: \newblock Clarendon, Oxford, 1962.
1479:
1480: \bibitem{nash:1950ef}
1481: J.~F. Nash.
1482: \newblock Equilibrium points in n-person games.
1483: \newblock {\em Proc. Natl. Acad. Sci. U.S.A.}, 36:48--49, 1950.
1484:
1485: \bibitem{neu:1992aa}
1486: H.~C. Neu.
1487: \newblock The {C}risis in {A}ntibiotic {R}esistance.
1488: \newblock {\em Science}, 257:1064--1073, 1992.
1489:
1490: \bibitem{nowak:2006bo}
1491: M.~A. Nowak.
1492: \newblock {\em Evolutionary Dynamics}.
1493: \newblock Harvard University Press, Cambridge, MA, 2006.
1494:
1495: \bibitem{nowak:2006pw}
1496: M.~A. Nowak.
1497: \newblock Five rules for the evolution of cooperation.
1498: \newblock {\em Science}, 314:1560--1563, 2006.
1499:
1500: \bibitem{nowak:1992pw}
1501: M.~A. Nowak and R.~M. May.
1502: \newblock Evolutionary games and spatial chaos.
1503: \newblock {\em Nature}, 359:826--829, 1992.
1504:
1505: \bibitem{nowak:2004pw}
1506: M.~A. Nowak, A.~Sasaki, C.~Taylor, and D.~Fudenberg.
1507: \newblock Emergence of cooperation and evolutionary stability in finite
1508: populations.
1509: \newblock {\em Nature}, 428:646--650, 2004.
1510:
1511: \bibitem{nowak:1990aa}
1512: M.~A. Nowak and K.~Sigmund.
1513: \newblock The evolution of stochastic strategies in the prisoner's dilemma.
1514: \newblock {\em Acta Appl. Math.}, 20:247--265, 1990.
1515:
1516: \bibitem{nowak:1998is}
1517: M.~A. Nowak and K.~Sigmund.
1518: \newblock Evolution of indirect reciprocity by image scoring.
1519: \newblock {\em Nature}, 393:573--577, 1998.
1520:
1521: \bibitem{nowak:2004aa}
1522: M.~A. Nowak and K.~Sigmund.
1523: \newblock Evolutionary dynamics of biological games.
1524: \newblock {\em Science}, 303:793--799, 2004.
1525:
1526: \bibitem{ohta:2002aa}
1527: T.~Ohta.
1528: \newblock Near-neutrality in evolution of genes and gene regulation.
1529: \newblock {\em Proc. Natl. Acad. Sci. USA}, 99:16134--16137, 2002.
1530:
1531: \bibitem{ohtsuki:2007aa}
1532: H.~Ohtsuki, P.~Bordalo, and M.~A. Nowak.
1533: \newblock The one-third law of evolutionary dynamics.
1534: \newblock {\em Jour. Theor. Biol.}, 249:289--295, 2007.
1535:
1536: \bibitem{ohtsuki:2006na}
1537: H.~Ohtsuki, C.~Hauert, E.~Lieberman, and M.~A. Nowak.
1538: \newblock A simple rule for the evolution of cooperation on graphs.
1539: \newblock {\em Nature}, 441:502--505, 2006.
1540:
1541: \bibitem{pitcher:1992aa}
1542: T. Pitcher.
1543: \newblock Who dares, wins - the function and evolution of predator inspection
1544: behavior in shoaling fish.
1545: \newblock {\em Neth. J. Zool.}, 42:371--391, 1992.
1546:
1547: \bibitem{rainey:2003an}
1548: P.~B. Rainey and K.~Rainey.
1549: \newblock Evolution of cooperation and conflict in experimental bacterial
1550: populations.
1551: \newblock {\em Nature}, 425:72--74, 2003.
1552:
1553: \bibitem{reichenbach:2007aa}
1554: T.~Reichenbach, M.~Mobilia, and E.~Frey.
1555: \newblock Mobility promotes and jeopardizes biodiversity in
1556: rock--paper--scissors games.
1557: \newblock {\em Nature}, 448:1046--1049, 2007.
1558:
1559: \bibitem{reichenbach:2007bb}
1560: T.~Reichenbach, M.~Mobilia, and E.~Frey.
1561: \newblock Noise and correlations in a spatial population model with cyclic
1562: competition.
1563: \newblock {\em Phys. Rev. Lett.}, 99:238105, 2007.
1564:
1565: \bibitem{sandholm:2007bo}
1566: W.~H. Sandholm.
1567: \newblock {\em Population games and evolutionary dynamics}.
1568: \newblock MIT Press, Cambridge, MA, 2007.
1569:
1570: \bibitem{sandholm:2007dy}
1571: W.~H. Sandholm and E.~Dokumaci.
1572: \newblock Dynamo: Phase diagrams for evolutionary dynamics. software suite.
1573: \newblock {\em http://www.ssc.wisc.edu/~whs/dynamo}, 2007.
1574:
1575: \bibitem{sato:2002le}
1576: Y.~Sato, E.~Akiyama, and J.~D. Farmer.
1577: \newblock Chaos in learning a simple two-person game.
1578: \newblock {\em Proc. Natl. Acad. Sci. USA}, 99:4748--4751, 2002.
1579:
1580: \bibitem{sato:2003le}
1581: Y.~Sato and J.~P. Crutchfield.
1582: \newblock Coupled replicator equations for the dynamics of learning in
1583: multiagent systems.
1584: \newblock {\em Phys. Rev. E}, 67:015206(R), 2003.
1585:
1586: \bibitem{saunders:1988aa}
1587: C.~D. Saunders and G. Hausfater.
1588: \newblock The functional significance of baboon grooming behavior.
1589: \newblock {\em Annals N.Y. Acad. Sci.}, 525(1):430--432, 1988.
1590:
1591: \bibitem{schuster:1995le}
1592: H.~G. Schuster.
1593: \newblock {\em Deterministic Chaos}.
1594: \newblock VCH, Weinheim, 3rd edition, 1995.
1595:
1596: \bibitem{selten:1990le}
1597: R.~Selten.
1598: \newblock Bounded rationality.
1599: \newblock {\em Journal of Institutional and Theoretical Economics},
1600: 146:649--658, 1990.
1601:
1602: \bibitem{semmann:2003he}
1603: D.~Semmann, H.~J. Krambeck, and M.~Milinski.
1604: \newblock Volunteering leads to rock-paper-scissors dynamics in a public goods
1605: game.
1606: \newblock {\em Nature}, 425(6956):390--393, 2003.
1607:
1608: \bibitem{sigmund:2007oz}
1609: K. Sigmund.
1610: \newblock Punish or perish? retaliation and collaboration among humans.
1611: \newblock {\em Trends Ecol. Evol.}, 22(11):593--600, 2007.
1612:
1613: \bibitem{sinervo:2006aa}
1614: B.~Sinervo, A.~Chaine, J.~Clobert, R.~Calsbeek, L.~Hazard, L.~Lancaster, A.G.
1615: McAdam, S.~Alonzo, G.~Corrigan, and M.E. Hochberg.
1616: \newblock Self-recognition, color signals, and cycles of greenbeard mutualism
1617: and altruism.
1618: \newblock {\em Proc. Natl. Acad. Sci. USA}, 103:7372--7377, 2006.
1619:
1620: \bibitem{sinervo:1996le}
1621: B.~Sinervo and C.~M. Lively.
1622: \newblock The rock-paper-scissors game and the evolution of alternative male
1623: strategies.
1624: \newblock {\em Nature}, 380:240--243, 1996.
1625:
1626: \bibitem{stammbach:1982aa}
1627: E. Stammbach and H. Kummer.
1628: \newblock Individual contributions to a dyadic interaction: An analysis of
1629: baboon grooming.
1630: \newblock {\em Anim. Behav.}, 30:964--971, 1982.
1631:
1632: \bibitem{storch:07}
1633: D. Storch, P.~A. Marquet, and J.~H. Brown, editors.
1634: \newblock {\em Scaling Biodiversity}.
1635: \newblock Ecological Reviews. Cambridge University Press, Cambridge, 2007.
1636:
1637: \bibitem{Szabo:2007aa}
1638: G.~Szab{\'{o}} and G.~F{\'{a}}th.
1639: \newblock Evolutionary games on graphs.
1640: \newblock {\em Physics Reports}, 446:97--216, 2007.
1641:
1642: \bibitem{szabo:2007jt}
1643: P. Szab\'{o}, T. Cz\'{a}r\'{a}n, and G. Szab\'{o}.
1644: \newblock Competing associations in bacterial warfare with two toxins.
1645: \newblock {\em J. theor. Biol.}, 248:736--744, 2007.
1646:
1647: \bibitem{szolnoki:2004aa}
1648: A. Szolnoki and G. Szab\'{o}.
1649: \newblock Phase transitions for rock-scissors-paper game on different networks.
1650: \newblock {\em Phys. Rev. E}, 70:037102, 2004.
1651:
1652: \bibitem{taylor:2004wv}
1653: C.~Taylor, D.~Fudenberg, A.~Sasaki, and M.~A. Nowak.
1654: \newblock Evolutionary game dynamics in finite populations.
1655: \newblock {\em Bull. Math. Biol.}, 66:1621--1644, 2004.
1656:
1657: \bibitem{taylor:1978wv}
1658: P.~D. Taylor and L.~Jonker.
1659: \newblock Evolutionary stable strategies and game dynamics.
1660: \newblock {\em Math. Biosci.}, 40:145--156, 1978.
1661:
1662: \bibitem{tilman:nature06}
1663: D. Tilman, P.~B. Reich, and J.~M.~H. Knops.
1664: \newblock Biodiversity and ecosystem stability in a decade-long grassland
1665: experiment.
1666: \newblock {\em Nature}, 441:629--632, 2006.
1667:
1668: \bibitem{traulsen:2005hp}
1669: A.~Traulsen, J.~C. Claussen, and C.~Hauert.
1670: \newblock Coevolutionary dynamics: From finite to infinite populations.
1671: \newblock {\em Phys. Rev. Lett.}, 95:238701, 2005.
1672:
1673: \bibitem{traulsen:2006hp}
1674: A.~Traulsen, J.~C. Claussen, and C.~Hauert.
1675: \newblock Coevolutionary dynamics in large, but finite populations.
1676: \newblock {\em Phys. Rev. E}, 74:11901, 2006.
1677:
1678: \bibitem{traulsen:2006aa}
1679: A.~Traulsen and M.~A. Nowak.
1680: \newblock Evolution of cooperation by multi-level selection.
1681: \newblock {\em Proc. Natl. Acad. Sci. USA}, 103:10952--10955, 2006.
1682:
1683: \bibitem{traulsen:2006bb}
1684: A.~Traulsen, M.~A. Nowak, and J.~M. Pacheco.
1685: \newblock Stochastic dynamics of invasion and fixation.
1686: \newblock {\em Phys. Rev. E}, 74:11909, 2006.
1687:
1688: \bibitem{traulsen:2006ab}
1689: A.~Traulsen, J.~M. Pacheco, and L.~A. Imhof.
1690: \newblock Stochasticity and evolutionary stability.
1691: \newblock {\em Phys. Rev. E}, 74:021905, 2006.
1692:
1693: \bibitem{traulsen:2007cc}
1694: A.~Traulsen, J.~M. Pacheco, and M.~A. Nowak.
1695: \newblock Pairwise comparison and selection temperature in evolutionary game
1696: dynamics.
1697: \newblock {\em J. Theor. Biol.}, 246:522--529, 2007.
1698:
1699: \bibitem{traulsen:2004iq}
1700: A.~Traulsen, T.~R{\"o}hl, and H.~G. Schuster.
1701: \newblock Stochastic gain in population dynamics.
1702: \newblock {\em Phys. Rev. Lett.}, 93:028701, 2004.
1703:
1704: \bibitem{traulsen:2008aa}
1705: A.~Traulsen, N.~Shoresh, and M.~A. Nowak.
1706: \newblock Analytical results for individual and group selection of any
1707: intensity.
1708: \newblock {\em Bull. Math. Biol.}, 70:1410--1424, 2008.
1709:
1710: \bibitem{trivers:1971hp}
1711: R.~L. Trivers.
1712: \newblock The evolution of reciprocal altruism.
1713: \newblock {\em The Quarterly Review of Biology}, 46:35--57, 1971.
1714:
1715: \bibitem{turner:1999hp}
1716: P.~E. Turner and L.~Chao.
1717: \newblock Prisoner's {D}ilemma in an {RNA} virus.
1718: \newblock {\em Nature}, 398:441--443, 1999.
1719:
1720: \bibitem{turner:2003hp}
1721: P.~E. Turner and L. Chao.
1722: \newblock Escape from prisoner's dilemma in {RNA} phage $\phi 6$.
1723: \newblock {\em Am. Nat.}, 161:497--505, 2003.
1724:
1725: \bibitem{neumann:1944ef}
1726: J.~von Neumann and O.~Morgenstern.
1727: \newblock {\em Theory of Games and Economic Behavior}.
1728: \newblock Princeton University Press, Princeton, 1944.
1729:
1730: \bibitem{wilson:1975sg}
1731: D.~S. Wilson.
1732: \newblock A theory of group selection.
1733: \newblock {\em Proc. Natl. Acad. Sci. USA}, 72:143--146, 1975.
1734:
1735: \bibitem{wilson:1994gs}
1736: D.~S. Wilson and E.~Sober.
1737: \newblock Reintroducing group selection to the human behavioral sciences.
1738: \newblock {\em Behavioral and Brain Sciences}, 17:585--654, 1994.
1739:
1740: \bibitem{zeeman:1980ze}
1741: E.~C. Zeeman.
1742: \newblock Population dynamics from game theory.
1743: \newblock {\em Lecture Notes in Mathematics}, page 819, 1980.
1744:
1745: \end{thebibliography}
1746:
1747: \end{document}
1748: