1: %% LyX 1.2 created this file. For more info, see http://www.lyx.org/.
2: %% Do not edit unless you really know what you are doing.
3: %\documentclass[12pt,english]{article}
4: \documentclass[aps,pre,twocolumn,floats,floatfix,english]{revtex4}
5: \usepackage[T1]{fontenc}
6: \usepackage[latin1]{inputenc}
7: \usepackage{amsmath}
8: \usepackage{graphicx}
9: \usepackage{setspace}
10: \usepackage{epsfig}
11: \makeatletter
12:
13: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%% LyX specific LaTeX commands.
14: \providecommand{\LyX}{L\kern-.1667em\lower.25em\hbox{Y}\kern-.125emX\@}
15:
16: \usepackage{babel}
17: \makeatother
18: \begin{document}
19: \title{Biology helps you to win a game}
20:
21: \author{Marko Sysi-Aho}
22: \author{Anirban Chakraborti}
23: \email{anirban@lce.hut.fi}
24: \homepage{http://www.lce.hut.fi/~anirban}
25: \author{Kimmo Kaski}
26:
27: \affiliation{Laboratory of Computational Engineering, Helsinki
28: University of Technology, \\
29: P. O. Box 9203, FIN-02015 HUT, Finland.}
30:
31: \begin{abstract}
32:
33: We present a game of interacting agents which mimics the complex dynamics found in many natural and social
34: systems. These agents modify their strategies periodically, depending on their
35: performances using genetic crossover mechanisms, inspired by biology. We study
36: the performances of the agents under different conditions, and how they
37: adapt themselves. In addition the dynamics of the game is investigated.
38:
39: \end{abstract}
40:
41: \maketitle
42: \section{Introduction}
43: Is the ``survival of the fittest'' principle limited to biology only? Perhaps
44: not and there could be other spheres of life in which this principle is
45: applicable. Competition plays a key role and in order to compete and thus survive in any
46: environment or situation, one primarily needs to adapt in order to succeed.
47: Then what is adaptation and evolution? Adaptation is an alteration or
48: adjustment in
49: structure or habits, often hereditary, by which a species or individual improves
50: its condition in relationship to its environment. Evolution is the change
51: in the genetic composition of a population during successive generations, as a
52: result of natural selection acting on the genetic variation among individuals,
53: and resulting in the development of a new species.
54: Here, we show that in the behaviour of various complex systems found in
55: natural and social environments \cite{parisi,huberman,nowak,lux,arthur}, that
56: can be characterized by
57: the competition among interacting agents for scarce resources,
58: adaptation to the environment plays a very important role.
59:
60:
61: These agents could be
62: diverse in form and in capability, ranging for example, from carcinogenic cells in the
63: human body to multinational firms in the global financial market. In these dynamically
64: evolving complex systems the nature of agents and their behaviour
65: differ a lot but they have a common underlying mechanism. In order to have a deeper
66: understanding of the interactions of the large number of agents, one
67: should first consider the individual capabilities of the agents. Its behaviour may
68: be thought of as a collection of simple rules governing ``responses'' to
69: numerous ``stimuli''.
70: The rules of action serve as the agents' strategies, and the
71: behaviour of an agent is the rules acting sequentially.
72: Therefore, in order to model any complex dynamically adaptive system, a major concern is the
73: selection and representation of the stimuli and responses, since the behaviour
74: and strategies of the component agents are determined thereby.
75: Then the agent needs to adapt to different situations, where the
76: experience of an agent guides it to change its structure so that as time
77: passes, the agent learns to make better use of the environment for its own
78: benefit.
79: However, the timescales over which the agents adapt vary from one individual to
80: another and also from one system to another.
81:
82:
83: In complex adaptive systems, many interesting temporal patterns are produced,
84: since a major part of the environment of a particular agent includes other
85: adaptive agents and a considerable amount of agent's effort goes
86: in adaptation and reaction to the other agents.
87: Thus the situation is considerably different and more complicated than in
88: game theory \cite{game} and conventional theories in economics, where the study is of patterns in behavioural equilibrium that
89: induce no further interaction.
90:
91:
92: In this paper, we study a simple game based on the basic minority game
93: \cite{challet1,challet2,cavagna,riolo,lamper}, where the agents adapt
94: themselves by modifying their strategies from time to time, depending on their
95: current performances, using genetic crossover mechanisms
96: \cite{holland,goldberg,lawrence,Marko1}. The game can be a very simple
97: representation of a complex adaptive system. We make a comparative study of
98: their performances with the various mechanisms and in a ``test'' situation.
99:
100: \section{Model}
101: In this section we give a brief description of the model. The basic minority
102: game consists of an odd number $N$ of agents who can perform at a given time
103: $t$, any of the two possible actions denoted here by $0$ or $1$. The
104: minority game was based on the El Farol bar problem, created by Brian Arthur,
105: in which a population of agents have to decide whether to go to the bar every
106: Thursday night, and so there were two possible actions ``to attend'' denoted by
107: $1$ and ``not to attend'' denoted by $0$, depending on whether the bar was too
108: crowded or not \cite{arthur}.
109: An agent wins the game if it is one of the members of the minority
110: group.
111: All the agents are assumed to have access to finite amount of {}``global''
112: information: a common bit-string {}``memory'' of the $M$ most recent
113: outcomes. With this there are $2^M$ possible ``history''
114: bit-strings. Now, a {}``strategy'' consists of two possible
115: responses, which in the binary sense are an action $0$ or
116: action $1$ to
117: each possible history bit-strings. Thus, there are $2^{2^{M}}$
118: possible strategies constituting the whole {}``strategy space''.
119:
120: Each time
121: the game has been played, time $t$ is incremented by unity and one {}``virtual'' point is assigned to the strategies that
122: predicted the correct outcome and the best strategy is the one which has
123: the highest virtual point score. The performance of a player is measured
124: by the number of times the player wins, and the strategy, which the
125: player uses to win, gets a {}``real'' point. The number of agents
126: who have chosen a particular action, say $1$ which represents ``to attend'',
127: is denoted by
128: $A_{1}(t)$ (also referred as ``attendance'') and it varies with time. We have plotted the attendance and performance
129: for the basic minority game in Fig. \ref{bmg}.
130:
131: \begin{figure}
132: \epsfig{file=figure1.eps,width=3.2in}
133: \caption{
134: Plots of (a) attendance and (b) performance of the players for the basic
135: minority game with $N=801$, $M=6$, $k=10$ and $T=5000$.
136: }
137: \label{bmg}
138: \end{figure}
139:
140:
141:
142: Now we define the total utility of the system as the number of persons in the
143: minority group at a given time $t$. For convenience, we mathematically define a scaled utility (total utility/maximum utility) as
144:
145: \begin{equation}
146: U=[(1-\theta(x_t-x_M))x_t+\theta(x_t-x_M)(N-x_t)]/x_M,
147: \end{equation}
148:
149: \noindent where $x_M=(N-1)/2$,
150: $x_t$ is either equal to $A_1(t)$ or $A_0(t)$, and $\Theta (x)$ is Heaviside step function:
151:
152: \begin{displaymath}
153: \theta(x)=\left\{ \begin{array}{ll}
154: 0 & \textrm{ when $x \le 0$} \\
155: 1 & \textrm{ when $x > 0$}.
156: \end{array}\right.
157: \end{displaymath}
158:
159:
160: The players examine their performances after every time interval $\tau $.
161: If a player finds that he is among
162: the fraction $n$ (where $0<n<1$) who are the worst performing
163: players, he adapts himself and modifies his strategies. The mechanism by which
164: the player creates new strategies is genetic crossover, whereby
165: he selects the two {}``parents'' from his pool of $k$ strategies and
166: creates two new {}``children'' \cite{lawrence, Marko1}, as described in
167: Fig. \ref{cross}.
168:
169:
170: \begin{figure}
171: \epsfig{file=figure2.eps,width=3.2in}
172: \caption{
173: Schematic diagram to illustrate the mechanism of one-point genetic
174: crossover for producing new strategies. The strategies $s_{i}$ and
175: $s_{j}$ are the parents. We choose the breaking point randomly and
176: through this one-point genetic crossover, the children $s_{k}$ and
177: $s_{l}$ are produced.
178: }
179: \label{cross}
180: \end{figure}
181:
182: If the parents are chosen randomly from the pool of strategies then the
183: mechanism represents a ``one-point genetic crossover'' and if the parents are the
184: best strategies then the mechanism represents a ``hybridized genetic crossover''.
185: The children may replace parents or two worst strategies and accordingly four different
186: interesting cases arise:
187: (a) one-point genetic crossover with parents ``killed'', i.e. parents are replaced by the children,
188: (b) one-point genetic crossover with parents ``saved'', i.e. the two worst
189: strategies are replaced by the children but the parents are retained,
190: (c) hybridized genetic crossover with parents ``killed''
191: and (d) hybridized genetic crossover with parents ``saved''.
192:
193: It should be noted that the mechanism of evolution
194: of strategies is considerably different from earlier attempts \cite{challet1,li1,li2}. This is because in this mechanism the strategies are changed by the agents
195: themselves and even though the strategy space evolves continuously, its size
196: and dimensionality remain the same.
197:
198:
199: The Hamming distance $d_H$ between two bit-strings is defined as the ratio of
200: the number of uncommon bits to the total length of the bit strings. It is a
201: measure of the correlation between two strategies:
202:
203: \begin{displaymath}
204: d_H=\left\{ \begin{array}{lll}
205: 0 & \textrm{ correlated} \\
206: 0.5 & \textrm{ uncorrelated}\\
207: 1 & \textrm{ anti-correlated}
208: \end{array}\right.
209: \end{displaymath}
210:
211: \noindent
212: which can be plotted as the game evolves.
213:
214:
215: \section{Results}
216:
217: In order to determine
218: which mechanism is the most efficient, we have made a comparative study of the
219: four cases, mentioned earlier. We plot the attendance as a function of time for
220: the different mechanisms in Fig. \ref{all4a}.
221:
222:
223: \begin{figure}
224: \epsfig{file=figure3.eps,width=3.2in}
225: \caption{ Plots of the attendances by choosing parents
226: randomly (a) and (b), and using the best parents in a player's pool
227: (c) and (d). In (a) and (c) case parents are replaced by children and
228: in (b) and (d) case children replace the two worst strategies.
229: Simulations have been done with
230: $N=801$, $M=6$, $k=16$, $t=40$, $n=0.4$ and $T=10000$.}
231: \label{all4a}
232: \end{figure}
233:
234: In Fig. \ref{all4b} we show the total utility of the system in each of the cases (a)-(d), where we
235: have plotted results of the average over 100 runs and each point in the utility
236: curve represents a time average taken over a bin of length 50
237: time-steps. The simulation time is
238: doubled from those in Fig. \ref{all4a}, in order to expose the asymptotic
239: behaviour better.
240: On the basis of Figs. \ref{all4a} and \ref{all4b}, we find that the case (d)
241: is the most efficient.
242:
243: \begin{figure}
244: \epsfig{file=figure4.eps,width=3.2in}
245: \caption{ Plots of the scaled utilities of the four different mechanisms in comparison with that of the basic minority game.
246: Each curve represents an ensemble average over 100 runs
247: and each point in a curve is a time average over a bin
248: of length 50 time-steps. In the inset, the quantity ($1-U$) is plotted against scaled time in the double logarithmic scale. Simulations are done with $N=801$, $M=6$, $k=16$,
249: $t=40$, $n=0.4$ and $T=20000$.
250: \label{all4b}}
251: \end{figure}
252:
253:
254:
255:
256: \begin{figure}
257: \epsfig{file=figure5.eps,width=3.2in}
258: \caption{
259: Plot of the average Hamming distance of all the strategies in a pool of a player
260: with time, where the player adapts using (a) one-point genetic crossover and (b)
261: hybridized genetic crossover, and in both cases the two
262: worst strategies are replaced by the children and the parents are also saved.
263: Each curve is an ensemble average over 20 runs.}
264: \label{ham1}
265: \end{figure}
266:
267: In Fig. \ref{ham1} (a) one can see the evolution of the average Hamming distance
268: of all the strategies of a player in a game, where the player adapts using
269: one-point genetic crossover and the two worst strategies are replaced by the children and the parents are also saved.
270: It should be noted that the Hamming distance
271: can change only when the worst strategies
272: are replaced by the children and the parents are saved, where the bits in a
273: strategy pool can change over time. Otherwise the bits in the pool of strategies
274: remain the same.
275: We observe that the curves tend to move downwards from around $0.5$ towards zero, which means that as the time
276: evolves, the correlation amongst the strategies increases and the strategies in the pool of a particular agent converges
277: towards one strategy. The nature of the curves depend a lot on the parameters of the game.
278: In Fig. \ref{ham1} (b) one can see the evolution of the average Hamming distance
279: of all the strategies of a player in the game, where the player adapts using
280: hybridized genetic crossover and the two worst strategies are replaced by the
281: children and the parents are also saved. Here too, the strategies in the pool of a
282: particular agent converges towards one strategy, and at a faster rate than with the
283: previous mechanism.
284: We observe that increasing memory $M$ does not change dramatically
285: the convergence rate, but as we increase the number of strategies in
286: the pools, the convergence slows down.
287:
288:
289:
290: \begin{figure}
291: \epsfig{file=figure6.eps,width=3.2in}
292: \caption{
293: Plot of the performance of the players where after $T=3120$ time-steps, six players begin
294: to adapt and modify their strategies: three using hybridized genetic crossover
295: mechanism and the other three using one point genetic crossover, where children
296: replace the parents.
297: Other players play the basic minority game all the time and do not adapt.
298: The simulations are done with $N=801$, $M=8$,
299: $k=16$, $n=0.3$, $t=80$, and $T=10000$.}
300: \label{mixed2}
301: \end{figure}
302:
303: In order to investigate what happens in the level of an individual agent, we
304: created a competitive surrounding-- ``test'' situation where
305: after $T=3120$ time-steps, six players begin
306: to adapt and modify their strategies such that three are using hybridized genetic crossover
307: mechanism and the other three one point genetic crossover, where children
308: replace the parents.
309: The rest of the players play the basic minority game. In this case it turns out that
310: in the end the best players are those who use the hybridized mechanism,
311: second best are those using the one-point mechanism, and the bad
312: players those who do not adapt at all.
313: In addition it turns out that the competition amongst the players who adapt using the hybridized genetic
314: crossover mechanism is severe.
315:
316:
317: \section{Conclusion}
318:
319: We can summarize our findings by stating that adaptation improves not only the individual player's
320: performance but also improves the total utility of the system. The best results
321: are found for the players who adapt and modify their strategies using the
322: hybridized genetic crossover mechanism and the children replace the two worst
323: strategies and the parents are saved. The mechanism of adaptation is very simple
324: and can be used to model different complex adaptive systems. It can also be
325: potentially developed to include other features like mutation. We can thus say
326: that in a way, ``biology helps you to win a game''.
327:
328:
329: \begin{acknowledgments}
330: This research was partially supported by the Academy of
331: Finland, Research Centre for Computational Science and Engineering,
332: project no. 44897 (Finnish Centre of Excellence Programme 2000-2005).
333: \end{acknowledgments}
334:
335: \begin{thebibliography}{10}
336: \bibitem{parisi}G. Parisi, \textit{Physica A} \textbf{263}, 557 (1999).
337: \bibitem{huberman}B. A. Huberman, P. L. T. Pirolli, J. E. Pitkow and R. M. Lukose, \textit{Science}
338: \textbf{280}, 95 (1998).
339: \bibitem{nowak}M. Nowak and R. May, \textit{Nature} \textbf{359}, 826 (1992).
340: \bibitem{lux}T. Lux and M. Marchesi, \textit{Nature} \textbf{397}, 498 (1999).
341: \bibitem{arthur}W. B. Arthur, \textit{Am. Econ. Rev.} \textbf{84}, 406 (1994).
342: \bibitem{game}R. Myerson, \emph{Game Theory: Analysis of Conflict}
343: (Harvard University Press, Cambridge, Massachusetts, 1991).
344: \bibitem{challet1}D. Challet and Y.-C. Zhang, \textit{Physica A} \textbf{246}, 407 (1997).
345: \bibitem{challet2}D. Challet, M. Marsili and R. Zecchina, \textit{Phys. Rev. Lett.} \textbf{84},
346: 1824 (2000).
347: \bibitem{riolo}R. Savit, R. Manuca and R. Riolo, \textit{Phys. Rev. Lett.} \textbf{82}, 2203 (1999).
348: \bibitem{cavagna}A. Cavagna, J. P. Garrahan, I. Giardina and D. Sherrington, \textit{Phys. Rev. Lett.} \textbf{83}, 4429 (1999).
349: \bibitem{lamper}D. Lamper, S. D. Howison and N. F. Johnson, \textit{Phys. Rev. Lett.}
350: \textbf{88}, 17902 (2002).
351: \bibitem{holland}J. H. Holland, \textit{Adaptation in Natural and Artificial Systems},
352: University of Michigan Press, Ann Arbor (1975).
353: \bibitem{goldberg}D. E. Goldberg, \textit{Genetic Algorithms in Search, Optimization
354: and Machine Learning}, Addison-Wesley, Reading, Massachusetts (1989).
355: \bibitem{lawrence}D. Lawrence (Ed.), \textit{Handbook of Genetic Algorithms}, Van Nostrand
356: Reinhold, New York (1991).
357: \bibitem{Marko1}M. Sysi-Aho, A. Chakraborti and K. Kaski, \emph{preprint
358: available at cond-mat/0209525} (2002); \textit{Physica A}, in press (2003).
359: \bibitem{li1}Y. Li, R. Riolo and R. Savit, \emph{Physica A} \textbf{276}, 234 (2000).
360: \bibitem{li2}Y. Li, R. Riolo and R. Savit, \emph{Physica A} \textbf{276}, 265 (2000).
361: \end{thebibliography}
362:
363: \end{document}
364: