1: \documentstyle[aps,multicol,psfig]{revtex}
2:
3: \draft
4:
5: \begin{document}
6:
7: \title{Self-organization in populations of competing agents}
8:
9: \author{Alexei V\'azquez$^{1,2}$}
10:
11: \address{$^1$ Abdus Salam International Center for Theoretical Physics\\
12: Strada Costiera 11, P.O. Box 586, 34100 Trieste, Italy}
13:
14: \address{$^2$ Department of Theoretical Physics, Havana University\\
15: San L\'azaro y L, Havana 10400, Cuba}
16:
17: \maketitle
18:
19: \begin{abstract}
20:
21: A population of heterogenous agents compeeting through a minority rule is
22: investigated. Agents which frequently loose are selected for evolution by changing
23: their strategies. The stationary composition of the population resulting for this
24: self-organization process is computed analytically. Results are compared with
25: numerical simulations of two different minority games and other analytical
26: treatments available in the literature.
27:
28: \end{abstract}
29:
30: \begin{multicols}{2}\narrowtext
31:
32: The mechanisms under which a population of compeeting agents self-organizes is a
33: problem which has gained a lot of imterest among the physics community in the last
34: few years \cite{arthur94,zhang97,johnson99,hulst99,ceva00,lo00,paczuski00}. As in
35: traditional statistical mechanism the main goal is to determine the average behavior
36: of the population based on the elementary rules which characterize the competition
37: among agents. Given the actual state of art most of the works have been devoted to
38: the characterization, by means of numerical experiments or analytical treatments, of
39: the different models proposed in the literature
40: \cite{arthur94,zhang97,johnson99,paczuski00}, while a general description is still
41: missed.
42:
43: This work is an attempt to extract some general features present in the subclass of
44: models where the performance of the agents is characterized by certain minority rule
45: \cite{arthur94,zhang97,paczuski00}. Agents are assumed to be heterogeneous and the
46: main goal is to obtain which is the final composition of the population if certain
47: evolution mechanism is introduced. The agents will be supposed not to be subscribed
48: to any regular lattice which rule out the existence of spatial correlations allowing
49: a mean field (MF) treatment.
50:
51: The work is organized as follows. First the general features of the class of models
52: under consideration are introduced. Based on these rules the MF rate equations
53: describing the evolution of the population are derived and the stationary solutions
54: are computed. These results are compared with numerical simulations of two
55: particular minority "games". Moreover, a comparison with the analytical results
56: already available for the model by Johnson {\em et al}
57: \cite{johnson99,hulst99,ceva00,lo00} is also drown. In all cases a very good
58: agreement is obtained.
59:
60: Considers a population of $N$ (odd) heterogeneous Boolean agents characterized by
61: certain property $x$. The Boolean nature of the agents restrict their action to only
62: two possibilities. An example is the Arthur-bar problem \cite{arthur94} in which
63: each agent attend or not a bar based on the past attendance to the bar. Another
64: example is the minority game introduced by Challet and Zhang \cite{zhang97} in which
65: the agents can either buy or sell. In both models a good choice of $x$ is the
66: probability that an agent takes different decisions in two different steps. A
67: different choice is taken in the model introduced by Johnson {\em et al}
68: \cite{johnson99}. In this case $x$ is the probability that an agent accepts the
69: decision suggested by its strategy or does the opposite. In general one make think
70: in $x$ as any property, or set of properties, which distinguish the different
71: strategies one agent can choose.
72:
73: On each step each agent take one of the two decisions and those being in the
74: minority win. The long time performance of the agents will be characterized by a
75: cumulative index $z$ in such a way that each time an agent win it is rewarded
76: with a positive point ($z\rightarrow z+1$), otherwise it is punished receiving a
77: negative point ($z\rightarrow z-1$). Evolution in the population is introduced
78: assuming that each time the performance $z$ of an agent goes below a threshold
79: $-z_c$ ($z_c>0$) it is selected for evolution an change its strategy: a new value
80: of $x$ ($x\rightarrow x^\prime$) is selected among certain distribution
81: $P_0(x^\prime)$ and its performance is reseted to zero ($z\rightarrow0$).
82:
83: A similar evolutionary mechanism has been already considered in \cite{johnson99}.
84: One also may think in other implementations like the extremal rules considered in
85: \cite{zhang97,paczuski00}, or a probabilistic rule in which agents can change
86: strategy even below $z_c$ according to certain probability, which in general may
87: depend on $z$. In any case it is expected that all of them give the same qualitative
88: behavior in the large $z_c$ limit.
89:
90: Having defined the model let us determine which is the stationary composition of the
91: population of agents $P(x)$, the fraction of agents of type $x$. For doing so we
92: first derive the rate equations which describe the dynamical evolution of $P(x)$,
93: which in general can be written as
94: \begin{equation}
95: \frac{\partial}{\partial t}P(x)=\sum_{x^\prime}\left[
96: W(x^\prime,x)P(x^\prime)-W(x,x^\prime)P(x)\right],
97: \label{eq:1}
98: \end{equation}
99: where $W(x^\prime,x)$ is the transition rate from $x^\prime$ to $x$.
100:
101: The only rule which allows a change on $x$ is the evolution rule, in which the agent
102: chose a new value of $x^\prime$, selected with probability $P_0(x^\prime)$. Thus,
103: if $\lambda(x)$ is the fraction of $x$-agents which are changing their strategy per
104: unit time then the transition rates are given by
105: $W(x,x^\prime)=\lambda(x)P_0(x^\prime)$. After substitution of this expression into
106: eq. (\ref{eq:1}) it results that
107: \begin{equation}
108: \frac{\partial}{\partial t}P(x)=P_0(x)\sum_{x^\prime}
109: \lambda(x^\prime)P(x^\prime)-\lambda(x)P(x).
110: \label{eq:2}
111: \end{equation}
112:
113: To determine $\lambda(x)$ we need to consider the temporal evolution of the
114: cumulative performance index $z(x)$. Let $P_m(x)$ be the probability per unit
115: time that a $x$-agent is in the minority in one step. Thus, which probability
116: $P_m(x)$ ($1-P_m(x)$) the performance increases (decreases) by one unit. More
117: generaly one can define $P_m(x)$ as the probability that an agent wins a point,
118: regardless how it did. On the other hand, if an agent changes strategy, which
119: happens with probability $\lambda(x)$, then its performance increases by $z_c$.
120: These elementary processes lead to the rate equation
121: \begin{equation}
122: \frac{\partial}{\partial t} z(x)=2P_m(x)-1+z_c\lambda(x).
123: \label{eq:3}
124: \end{equation}
125:
126: Provided $P_m(x)<1/2$ for all $x$ the system described by eqs. (\ref{eq:2}) and eq.
127: (\ref{eq:3}) will always reach a stationary state. In this state time derivatives
128: vanish obtaining
129: \begin{equation}
130: \lambda(x)=\frac{1}{z_c}[1-2P_m(x)],
131: \label{eq:4}
132: \end{equation}
133: \begin{equation}
134: P(x)=\frac{A}{1-2P_m(x)}P_0(x),
135: \label{eq:5}
136: \end{equation}
137: where $A$ is a normalization constant.
138:
139: Agents with $P_m(x)$ close to $1/2$ are more stable to change in strategy as it can
140: be seen from eq. (\ref{eq:4}) and, therefore, they will have a larger participation
141: ratio in the stationary population, as follows from eq. (\ref{eq:5}). The
142: distribution from which the value of $x$ is extracted is thus modulated by the
143: probability of being in the minority, yielding the stationary population in eq.
144: (\ref{eq:5}).
145:
146: On the other hand, from eq. (\ref{eq:4}) it can be seen that in the limit
147: $z_c\rightarrow\infty$ the fraction of agents changing strategy becomes
148: infinitesimal and, therefore, in this limit one expect to obtain the same results as
149: if one use an extremal evolution rule. Another aspect to be emphasized is the fact
150: that the parameter $z_c$, which in principle is the only evidence of the particular
151: evolution rule chosen here, does not appears in the expression for the stationary
152: distribution of agents in eq. (\ref{eq:5}) and, therefore, this result is expected
153: to hold independent of the particular evolution rule chosen.
154:
155: So far the results obtained here are very general and expected to apply to a wide
156: class of models of Boolean agents with minority rule. To go further we have to
157: determine $P_m(x)$ which may however depends on the particular model under
158: consideration. Below two different cases are analyzed. One is a very simple model of
159: non-interacting agents (NIA) which make their decision at random. The other is the
160: very recent implementation of a population of Boolean agents build up onto a
161: Kauffman's network (AKN) introduced by Paczuski and Bassler \cite{paczuski00}.
162:
163: In the AKN model introduced in \cite{paczuski00} agents makes their decisions based
164: on the previous decision of some other agents. A Boolean variable $\sigma_i$
165: ($i=1,2,\ldots,N$; $\sigma_i=0,1$) is assigned to each agent which is representative
166: of the two possible decisions one agent may take. The decision taken by each agent
167: is based on the decision took by other $K$ agents chosen at random
168: ($i_1,\ldots,i_K$) in the previous step, according to certain Boolean function $f_i$
169: selected at random among the set of all the $2^{2^K}$ possible Boolean functions
170: with $K$ inputs, i.e.
171: \begin{equation}
172: \sigma_i(t+1)=f_i[\sigma_{i_1}(t),\ldots,\sigma_{i_K}(t)].
173: \label{eq:6}
174: \end{equation}
175: Without lost of generality and for a reason which becomes clear below, Booleans
176: functions which give the same output independent of the inputs are rule out.
177:
178: In the original variant of this model \cite{paczuski00} an extremal evolution rule
179: is used, such that after certain time the worst agent is selected for evolution.
180: This rule is relaxed here with the barrier evolution rule considered above, which
181: yields the extremal dynamics as $z_c\rightarrow\infty$. Moreover, one can see that a
182: property that makes differences among the agents can be the number of times 1
183: appears on the $M=2^K$ outputs of its Boolean function, denoted by $n$. If $n$ is
184: close to $M/2$ the agent will give as output 0 or 1, depending on the configuration
185: of its $K$ neighbors, with approximately the same probability. Otherwise, if $n$
186: is close to 1 or $M$ the agent will practically give the same output independent of
187: the configuration of its neighbors.
188:
189: The NIA model is a simplification of the above model in which interactions among
190: agents are rule out. In this case on each step an $n$-agent gives 1 as output with a
191: probability $n/M$ and 0 otherwise. For this case the agent's decision does not
192: depends on decisions of any other agents and, therefore, there is no other
193: correlation than the one introduced by the minority rule which depends on the output
194: of all agents.
195:
196: Numerical simulations were performed for a population of $N=99$ agents and
197: $K=2,3,4,5$. In all cases the system is updated until it reaches the stationary
198: state and then average is taken over the temporal evolution of the population. The
199: resulting data is averaged over different realizations of the initial Boolean
200: functions and over the choice of neighbors in the interacting case.
201:
202: Since the new Boolean functions are selected at random one has
203: \begin{equation}
204: P_0(n)=B(n;0.5,M)/\sum_{r=1}^{M-1}B(r;0.5,M),
205: \label{eq:6a}
206: \end{equation}
207: where $B(n;p,m)$ is the Binomial distribution, the probability to obtain 1 $n$ times
208: and $0$ $m-n$ times given on each single event 1 happens with probability $p$. The
209: values $n=0$ and $M$ are rule out because they give fixed strategies and
210: consequently the Binomial distribution is renormalized.
211:
212: To compute the stationary composition $P(n)$ one has to compute the probability
213: $P_m(n)$ that an agent of type $n$ is in the minority, and then plug in the result
214: in eq. (\ref{eq:5}). For $N$ large the game is expected to be symmetric in the sense
215: that with probability $1/2$ the minority is the group of agents which takes the
216: 1 (or 0) as output. In such a case a fixed agent will, in a long time window, have a
217: probability $1/2$ of being in the minority while agents changing their output very
218: often are expected to has a lower probability to be in the minority.
219:
220: The main hypothesis taken here is that $1-2P_m(n)=f[\rho(n)]$, where $f[\rho(n)]$ is
221: a smooth function of the probability per unit time $\rho(n)$ that an agent change
222: its output. Since fixed players ($\rho=0$) has a probability $1/2$ to be in the
223: minority then $f(0)=0$. In the following $f(\rho)$ is expanded around $\rho=0$,
224: keeping only the linear term, resulting
225: \begin{equation}
226: 1-2P_m(n)\propto \rho(n)+{\cal O}[\rho(n)^2].
227: \label{eq:7}
228: \end{equation}
229:
230: Now, for NIA $\rho(n)$ is just the probability to find two different outputs on the
231: agent's strategy, which is given by $P_c(n)=2(n/M)(1-n/M)$. For AKN one should also
232: take into account that agents can changes their output only if at least one of its
233: inputs has changed output in the previous step. Within a MF approximation the last
234: event happens with probability $K\bar{\rho}$. Thus, in general
235: \begin{equation}
236: \rho(n)=BP_c(n),
237: \label{eq:8}
238: \end{equation}
239: where $B=1$ and $B=K\bar{\rho}$ for NIA and AKN, respectively.
240:
241: By construction in both models $P_c(n)>0$ because the cases $n=0$ and $M$ have been
242: rule out. Hence, $1-2P_m(n)\propto BP_c(n)$ can only be zero if $B=0$ and if does it
243: is zero for all $n$. If the second possibility happens then all agents will have the
244: same probability to be in the minority and, therefore, their distribution in the
245: stationary state will be the distribution from where the Boolean functions are
246: extracted, i.e. $P(n)=P_0(n)$. Hence, eqs. (\ref{eq:5}),(\ref{eq:7})
247: and(\ref{eq:8}) yields the following alternative
248: \begin{equation}
249: P(n)=\left\{
250: \begin{array}{ll}
251: \frac{\bar{P_c}}{P_c(n)}P_0(n) & \text{for} B>0\\
252: P_0(n) & \text{for} B=0,
253: \end{array}
254: \right.
255: \label{eq:9}
256: \end{equation}
257: where $\bar{P_c}=\sum_nP_c(n)P(n)$, which is the final output of the present
258: calculation.
259:
260: For NIA as mentioned above $B=1$ and, therefore, the alternative $B>0$ takes place.
261: The comparison of this prediction with numerical data is shown in fig. \ref{fig:1}.
262: The agreement is quite well for all values of $K$, proving that the ansatz in eq.
263: (\ref{eq:8}) applies for this model.
264:
265: For AKN $B=K\bar{\rho}$ and one should analyze weather $\bar{\rho}$ is zero or not.
266: As it is well known the Kauffman's network display qualitative different behavior
267: depending on the value of $K$ \cite{kauffman93}. For $K\leq2$ and independent of the
268: initial conditions the network evolve to a frozen configuration with
269: $\bar{\rho}=0$. On the contrary for $K>2$ the network evolves to periodic orbits
270: with period growing exponentially with $N$ ("chaotic phase"), in which
271: $\bar{\rho}>0$.
272:
273: Thus, the population of agents build up onto the Kauffman's network is a very good
274: scenario to test the validity of eq. (\ref{eq:9}) because both alternatives can be
275: observed. The comparison is shown in fig. \ref{fig:1}. For $K=2$ it can seen that
276: the numerical data is in better agreement by $P(n)=P_0(n)$ as predicted above. On
277: the contrary for $K>2$ the data is in better agreement with the case $B>0$ in eq.
278: (\ref{eq:9}).
279:
280: For $K>2$ both the NIA and AKN models yields the same distribution $P(n)$ and,
281: therefore, the correlations introduced by the network are in those cases irrelevant.
282: Moreover with increasing $K$ the composition of the population gradually approaches
283: $P_0(n)$ which explains the lost of self-organization observed in \cite{paczuski00}.
284:
285: Finally the model of Johnson {\em et al} \cite{johnson99} is considered. In this
286: case the decisions taken by the agents depends on the previous history of the
287: winning group. For each agent it is available the information of which has been the
288: winning group $\sigma_w(t)$ ($\sigma_w(t)=0,1$) in the last $K$ steps. Moreover, to
289: each of then and strategy $f_i$is assigned, which is extracted at random among all
290: the possible Boolean functions of $K$ inputs. Then, for each possible past history
291: each agent will give a well defined output based on its own strategy, i.e.
292: \begin{equation}
293: \sigma_i(t+1)=f_i[\sigma_W(t),\ldots,\sigma_W(t-K+1)].
294: \label{eq:10}
295: \end{equation}
296: Notices that in this case the output of each agent depends on a global information
297: and not on the outputs of any other agent as in the AKN model (see eq.
298: (\ref{eq:6})).
299:
300: So far this model is just a variant of the minority game of Challet and Zhang
301: \cite{zhang97} for the case in which each agent has only one strategy at his
302: disposal. In the variation by Johnson {\em et al} \cite{johnson99} a probability
303: $p_i$ is assigned to each agent based on which the he accepts or not the output
304: of its strategy. With probability $p_i$ ($1-p_i$) he use the outcome (the opposite
305: outcome) of its strategy. Moreover, they introduced the cumulative index $z_i$ to
306: measure the performance of each agent in a similar way as described above. The only
307: difference is that for this model when $z_i$ goes below the threshold $-z_c$ the
308: agent does not change its Boolean function $f_i$ but rather its probability $p_i$,
309: choosing a new one at random in the interval $[p_i-R/2,p_i+R/2]$ with reflective
310: boundary conditions.
311:
312: Some analytical results are already available for this model
313: \cite{hulst99,lo00}. Using a diffusion like approach \cite{hulst99} or a detailed
314: probabilistic calculation \cite{lo00} it has been shown that in the stationary
315: population has the following composition
316: \begin{equation}
317: P(p)=\frac{A}{1-2P_m(p)},
318: \label{eq:11}
319: \end{equation}
320: where $A$ is a normalization constant and $P_m(p)$ is the probability that an agent
321: of type $p$ is in the minority \cite{note1}.
322:
323: Eq. (\ref{eq:11}) is actually quite similar to eq. (\ref{eq:5}), with the choice
324: $x=p$. Since for this model the new values of $p$ are extracted from a uniform
325: distribution it is expected that $P_0(p)$ does not depend on $p$. Hence, eq.
326: (\ref{eq:11}) can be seen as a limiting case of eq. (\ref{eq:5}) when applied to
327: the particular evolution rule of the model by Johnson {\em et al}
328: \cite{johnson99} where $P_0(x)=const.$.
329:
330: In order to go beyond this result one has to determine $P_m(p)$. This has already
331: been done in \cite{lo00} resulting that for large $N$ $1-2P_m(p)\approx C(N)2p(1-p)$
332: where $C(N)\sim N^{-1/2}$. This results does not seems to have any relation with the
333: ansatz in eq. (\ref{eq:9}). However, one should notice that $2p(1-p)$ is just the
334: probability $P_c(p)$ that an agent of type $p$ give two different outputs, given the
335: output of its Boolean function has remained fixed. Therefore, from this result
336: and eq. (\ref{eq:11}) it follows that
337: \begin{equation}
338: P(p)=\frac{\bar{P_c}}{P_c(p)}.
339: \label{eq:12}
340: \end{equation}
341: This eq. is just the $B>0$ alternative of eq. (\ref{eq:9}). Hence, the functional
342: dependence of $P_m(x)$ and $P(x)$ on $P_c(x)$ appears to be universal for the class
343: of models studied here.
344:
345: In summary, a heterogenous populations of compeeting agents has been studied. Based
346: on general arguments, such as the minority rule and evolution, the stationary
347: composition of the population of agents has been computed as a function of the
348: probability $P_m(x)$ of being in the minority. Further analysis reveals that the
349: relation $1-2P_m(x)\propto P_c(x)$ is universal for the class of model, where
350: $P_c(x)$ is the probability that an agent of type $x$ gives different outputs.
351:
352: For the AKN it is concluded that except for $K=2$ the correlations introduced by
353: the network are irrelevant and agents can be considered to be independent. In the
354: particular case $K=2$, which correspond to the critical network, the population
355: build-up onto the Kauffman's network reach an stationary state in which all
356: agents have a probability $1/2$ to be or not in the minority. Moreover, the lost of
357: self-organization with increasing $K$ was shown to takes place gradually.
358:
359: \section*{Acknowledgements}
360:
361: I thanks M. Paczuski for for useful comments and discussion. I also thanks Y.-C.
362: Zhang and M. Marsili for reading the manuscript.
363:
364: \begin{thebibliography}{99}
365:
366: \bibitem{arthur94} W. B. Arthur, Am. Econ. Rev. {\bf 84}, 406 (1994).
367:
368: \bibitem{zhang97} D. Challet and Y.-C. Zhang, Physics A {\bf 246}, 407 (1997);
369: {\em ibid} {\bf 256}, 514 (1998).
370:
371: \bibitem{johnson99} N. F. Johnson, P. M. Hui, R. Johnson, and T. S. Lo, Phys. Rev.
372: Lett. {\bf 82}, 3360 (1999)
373:
374: \bibitem{hulst99} R. D'hulst and G. J. Rodgers, Physica A {\bf 270}, 514 (1999).
375:
376: \bibitem{ceva00} H. Ceva, Physica A {\bf 277}, 496 (2000)
377:
378: \bibitem{lo00} T. S. Lo, P. M. Hui, and N. F. Johnson, cond-mat/0003379.
379:
380: \bibitem{paczuski00} M. Paczuski, K. E. Bassler, and \'A. Corral, Phys. Rev. Lett.
381: {\bf 84}, 3185 (2000).
382:
383: \bibitem{kauffman93} S. A. Kauffman, {\em The origin of order} (Oxford University
384: Press, New York, 1993).
385:
386: \bibitem{note1} A different notation is used in \cite{hulst99,lo00} in which
387: $P_m(p)$ is written as $\tau(p)$.
388:
389: \end{thebibliography}
390:
391: \begin{figure}
392: \centerline{\psfig{file=fig1.ps,width=3.5in,angle=-90}}
393: \caption{Stationary composition of the population of agents. The points where
394: obtained from numerical simulations of a population of $N=99$ agents with a
395: threshold $z_c=100$ and $K=2$ (squares), $K=3$ (circles) and $K=4$ (triangles). In
396: all cases the open and full symbols corresponds to the NIA and AKN models,
397: respectively. The solid lines are obtained using eq. (\ref{eq:9}) with $B=0$
398: (dashed lines) and $B>0$ (solid lines) for the corresponding values of $K$. }
399: \label{fig:1}
400: \end{figure}
401:
402: \end{multicols}
403:
404: \end{document}
405: