1:
2: \documentstyle[12pt,psfig]{article}
3: %\renewcommand{\baselinestretch}{2}
4: \begin{document}
5:
6: \title{Numerical and Theoretical Studies of Noise Effects in the
7: Kauffman Model}
8: \author{X. Qu\footnote{e-mail:quxh@uchicago.edu}, M. Aldana, Leo P. Kadanoff}
9: %\date{\today}
10: \maketitle
11: \begin{center}
12: The James Franck Institute, The University of Chicago.\\
13: 5640 South Ellis Avenue, Chicago, Il, 60637.
14: \end{center}
15:
16: \begin{abstract}
17: In this work we analyze the stochastic dynamics of the Kauffman model
18: evolving under the influence of noise. By considering the average
19: crossing time between two distinct trajectories, we show that
20: different Kauffman models exhibit a similar kind of behavior, even
21: when the structure of their basins of attraction is quite
22: different. This can be considered as a robust property of these
23: models. We present numerical results for the full range of noise
24: level and obtain approximate analytic expressions for the above
25: crossing time as a function of the noise in the limit cases of small
26: and large noise levels.
27: \end{abstract}
28:
29: \noindent {\bf Key words}: Kauffman Model, Noise, Crossing Time,
30: Robustness.
31:
32: \noindent {\bf Running title}: Kauffman Model with Noise.
33:
34: \pagebreak
35:
36: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
37: \section{Introduction}
38:
39: The Kauffman model (or $N$-$K$ model) describes the dynamics of a
40: network of $N$ Boolean spins, each controlled by $K$ other spins
41: through a binary function. It was first proposed by S. A. Kauffman in
42: 1969~\cite{69.1} as a model for cell differentiation and
43: genetic networks. Since then, its application has been extended to
44: many other fields in physics, biology, computational and social
45: sciences. During the past few decades, most of the work done on
46: Kauffman models has been dedicated to the study of the configuration
47: space structure, the length and number of cycles, the size of basins
48: of attraction, and the phase transition between ordered and disordered
49: phases (for references see~\cite{02.3}). These properties are
50: obtained by considering the deterministic dynamics of the system,
51: which is well known by now. However, those studies have shown that
52: some of the generic properties of the Kauffman model are far from
53: being robust. The non-robustness of the deterministic dynamics is
54: reflected, for example, in the fact that by slightly changing a given
55: initial configuration of spins, the system may ``jump'' from one basin
56: of attraction to a very different one. On the other hand, due to the
57: exponential growth of the state space with $N$, it is often necessary
58: to thoroughly probe the state space in order to determine a generic
59: property of the system, such as the mean number of different basins of
60: attraction or the mean cycle length. Actually, it has recently been
61: shown that a systematic bias due to an under-sampling of the state
62: space can be present in some of the results reported in the literature
63: during the last 30 years~\cite{01.7}. Therefore it is
64: valuable to find a method which reveals the robust properties of the
65: Kauffman model.
66:
67: Real networks are always subjected to external fluctuations.
68: Consequently, the relevant properties characterizing the network
69: should exhibit a certain degree of robustness to external
70: perturbations. In 1989, both Miranda \emph{et al}.~\cite{89.3} and
71: Golinelli \emph{et al}.~\cite{89.5} analyzed the stochastic dynamics
72: of the Kauffman model in the case in which an external noisy signal is
73: present. In this work we extend the study of the stochastic dynamics
74: of the Kauffman network with noise, by performing more accurate
75: numerical simulations as well as analytic calculations. We focus our
76: attention on the time it takes for two trajectories, starting out from
77: different initial conditions, to cross. We consider two cases. First
78: the situation in which each one of the $N$ spins is determined by $K$
79: other spins chosen randomly from everywhere in the system (the
80: Kauffman net). The second case is a d-dimensional lattice in which
81: each spin is preferentially coupled to its immediate neighbors. As we
82: will see, both models exhibit qualitatively similar behavior.
83:
84: In section \ref{sec:model}, we introduce the Kauffman model with
85: deterministic and stochastic dynamics. In section
86: \ref{sec:numerical} we describe our numerical results for both
87: Kauffman nets and lattices. In section \ref{sec:theoretical}, we
88: study closely the behavior of these models in the limits of small
89: and large noise. We summarize this work in section
90: \ref{sec:conclusions} with a brief discussion of the results.
91:
92: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
93: \section{The Kauffman model}\label{sec:model}
94:
95: %=============================================================
96: \subsection{Deterministic dynamics}
97: %=============================================================
98:
99: A Kauffman model consists of $N$ Boolean spins $\{S_1,S_2,\dots,S_N\}$
100: with $S_i$ being either zero or one. The value of each spin $S_i$ at
101: time $t+1$ is determined by the values of $K$ other spins
102: $S_{i_1},S_{i_2},\dots, S_{i_K}$, which are called the {\em
103: controlling elements} for spin $S_i$. (The number $K$ is called the
104: \emph{connectivity} of the system.) Once the connections in the system
105: are established, each spin $S_i$ is assigned with a Boolean function
106: $f_i$ of its $K$ controlling elements. A realization of the Kauffman
107: model consists of the set of connections and Boolean functions
108: assigned to every spin. The dynamics of the network is then given by
109: %
110: \begin{equation}
111: S_i(t+1)=f_i(S_{i_1}(t),S_{i_2}(t),\ldots,S_{i_K}(t))\ \mbox{for
112: $i=1,\ldots,N$}. \label{eq:definition}
113: \end{equation}
114: %
115: For convenience, we will denote by $\Sigma_t$ the state of the
116: system at time $t$:
117: %
118: \[
119: \Sigma_t=\{S_1(t),S_2(t),\dots,S_N(t)\}.
120: \]
121: %
122: In different Kauffman models, the assignments of the $K$ controlling
123: spins $S_{i_1},S_{i_2},\dots, S_{i_K}$ of each spin $S_i$ and the
124: dynamic rules $f_i$, are different. In \emph{Kauffman nets}, the
125: controlling elements of $S_i$ are assigned randomly, whereas in a
126: \emph{Kauffman lattice} they are chosen only among its nearest neighbors. The
127: dynamic rules $f_i$ are chosen randomly in such a way that its two
128: possible outcomes, 0 and 1, occur with probability $\rho$ and $1-\rho$
129: respectively. If the realization of the network is time-independent,
130: the network is called \emph{quenched}, while if either the set of
131: connections or the set of Boolean functions $f_i$ are re-assigned at
132: every time step, the network is termed \emph{annealed}.
133:
134: Annealed models are more convenient for theoretical studies than
135: quenched models. For example, by using the annealed approximation it
136: has been shown analytically that Kauffman nets exhibit three different
137: phases: frozen, critical and chaotic, depending upon the values of the
138: parameters $K$ and $\rho$ \cite{86.3}. The critical value of the
139: connectivity is given by $K_c=[2\rho(1-\rho)]^{-1}$. For $K<K_c$ the
140: system is in the frozen phase, whereas if $K>K_c$ it is in the chaotic
141: phase. Throughout this work we will use $\rho=1/2$, for which
142: $K_c=2$.
143:
144: But for most real cases (neural networks, genetic networks, etc.),
145: quenched models will be more appropriate since in real networks
146: neither the connections nor the interactions between the elements
147: change randomly at every moment. However, it has been shown that in
148: the limit $N\rightarrow\infty$, both the quenched and the annealed
149: Kauffman nets are exactly equivalent with respect to the evolution of
150: the overlap between different configurations, although not with
151: respect to the configurations themselves
152: \cite{86.1,87.3,87.6}. In this paper, our main focus is on quenched
153: Kauffman nets and lattices.
154:
155: Due to the finite size of the system, there are a finite number of
156: possible configurations, to wit $\Omega = 2^N$. Therefore,
157: starting out with an initial configuration, the system will
158: eventually fall into a previously visited state, after which the
159: same sequence of states repeatedly occurs again. The state space
160: breaks up into a multitude of cycles (or attractors). The totality
161: of points which end up in the same attractor represents its
162: \emph{basin of attraction}.
163:
164: %===============================================================
165: \subsection{Stochastic dynamics}
166: %===============================================================
167:
168: There are different ways of introducing noise into Kauffman models and
169: they reveal different features of the configuration space of the
170: model. Following Miranda and Parga \cite{89.3}, we introduced
171: noise in the following way:
172: %
173: \begin{equation}
174: S_i(t+1)=\left\{
175: \begin{array}{ll}
176: f_i(S_{i_1}(t),S_{i_2}(t),\ldots,S_{i_K}(t))&\mbox{with probability $1-r$},\\
177: 1-f_i(S_{i_1}(t),S_{i_2}(t),\ldots,S_{i_K}(t))&\mbox{with
178: probability $r$}.
179: \end{array}
180: \right. \label{eq:noisedefinition}
181: \end{equation}
182: %
183: In this way, every spin $S_i$ has a probability $r$ of violating
184: the deterministic rule~(\ref{eq:definition}). We will say that an
185: \emph{$n$-spin flip event} has occurred at a particular time step,
186: if $n$ spins violated the deterministic rule in this time step.
187: Notice that this stochastic dynamic rule has a symmetry about the
188: point $r=0.5$. For $r>0.5$, if we make a substitution $f_i\rightarrow
189: 1-f_i$, the rule becomes identical with the case of $1-r$. Since the
190: Boolean functions $f_i$ are assigned randomly, $f_i$ and $1-f_i$ are
191: equally likely to appear in a particular realization of the
192: model. After averaging over different realizations, the cases with
193: probabilities $r$ and $1-r$ are indeed identical. Due to this symmetry
194: in the stochastic dynamical rule~(\ref{eq:noisedefinition}), we only
195: need to consider the case $r\in[0,0.5]$. Note that for the particular
196: value $r=0.5$, $S_i(t+1)$ is equally likely to be zero or one
197: independently of the value of $f_i$.
198:
199: In the presence of noise, the concept of ``attractor'' does not
200: hold any more, since as the system evolves, there is a non-zero
201: probability of ``jumping'' to a different attractor, and
202: consequently every point in the state space can be reached from
203: any initial condition. In this sense, the ``boundaries'' between
204: different attractors become more diffuse as the level of noise
205: increases \cite{87.5}. However, we shall argue that the system
206: has a sort of effective attractor even in the presence of noise,
207: specifically when $r$ is large.
208:
209: One of the interesting things to study is the time it takes for
210: two trajectories to cross. Suppose that we start with two
211: different initial configurations, $\Sigma_0$ and
212: $\tilde{\Sigma}_0$, and let them evolve according to
213: (\ref{eq:noisedefinition}), noting all the configurations
214: produced:
215: %
216: \begin{eqnarray}
217: \Sigma_0&\rightarrow&\Sigma_1\rightarrow\Sigma_2\rightarrow\dots
218: \rightarrow\Sigma_\tau\nonumber\\
219: \tilde{\Sigma}_0&\rightarrow&\tilde{\Sigma}_1\rightarrow\tilde{\Sigma}_2
220: \rightarrow\dots\rightarrow\tilde{\Sigma}_\tau\nonumber
221: \end{eqnarray}
222: %
223: The crossing time $\tau$ is then defined as the time for which either
224: one of the trajectories coincides for the first time with a
225: configuration previously visited by the other trajectory. For example,
226: when $\tilde{\Sigma}_\tau$ is equal to any of the configurations
227: $\Sigma_0,\Sigma_1,\Sigma_2,\dots,\Sigma_\tau$. Two important cases
228: have to be distinguished, when $\Sigma_0$ and $\tilde{\Sigma}_0$
229: belong to the same basin of attraction, and when they belong to
230: different basins of attraction. For those cases we will denote the
231: crossing time by $\tau_s$ and $\tau_d$, respectively.
232:
233: Miranda and Parga examined the behavior of the system by considering
234: only the attractors with largest and next-largest basins of
235: attraction. They then showed that for small values of $r$, the
236: behavior of $\tau_s$ and $\tau_d$ are very different. At $r=0$, two
237: trajectories from the same basin will cross in a time comparable with
238: the sum of two times: first, the transient time required to enter the
239: attractor, and second, the length of the attractor itself. Conversely,
240: two trajectories starting out from different basins will never
241: cross. On the other hand, they found that for sufficiently large
242: values of $r$, the crossing time became independent of the starting
243: point. It did not matter where the two trajectories start, the two
244: basins merge into a sort of effective attractor and the trajectory
245: bounces around within that subset of the system-states. For these
246: larger values of $r$, the observed effective basin size increased with
247: $r$. From this, they drew the conclusion that the disappearance of
248: basins of attraction with the increase of $r$ is a sort of
249: hierarchical process in the sense that in a finite period of time, the
250: portion of the whole state space explored by a trajectory starting out
251: from a given basin of attraction increases with $r$. Complete
252: randomness is achieved at $r=0.5$, where the trajectory explores the
253: entire state space.
254:
255: As we will see, our simulation will show the same general behavior as
256: described by Miranda and Parga. But we shall explore the behavior in
257: more detail, showing the crossing time for the whole range of values
258: of $r$ and $K$.
259:
260: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
261: \section{Numerical results}\label{sec:numerical}
262:
263: Kauffman nets and Kauffman lattices differ in the structure of their
264: basins of attraction. One would expect this difference to be
265: reflected in the response of these models to the influence of
266: noise. For random realizations of the coupling functions $f_i$, what
267: determines the basin structure is the connectivity $K$. Therefore, we
268: will first analyze separately the cases with large $K$ (chaotic phase)
269: and small $K$ (ordered phase).
270:
271: We will partially follow Miranda and Parga's approach in that we
272: compute the average crossing time $\tau_s$ by using two initial
273: configurations, $\Sigma_0$ and $\tilde{\Sigma}_0$, in the largest
274: basin of attraction. For the average crossing time $\tau_d$, we pick
275: one starting configuration in the largest basin and the other in the
276: next largest one. The reasons to choose only the two largest basins
277: of attraction will be clear in what follows.
278:
279:
280: %=======================================================
281: \subsection{Kauffman models with large $K$}
282: %=======================================================
283:
284: We want first to characterize the structure of the basins of
285: attraction. One way of doing it is by computing the distribution of
286: basin-sizes $W(n)$, which is the fraction of the state space $\Omega$
287: occupied by the $n$-th largest basin of attraction. In
288: Fig.~\ref{fig:basin_size_largeK} we show $W(n)$ for a Kauffman net and
289: a 1-dimensional Kauffman lattice, both with $N=20$ and $K=5$. It can
290: be seen from this figure that both models exhibit a very similar
291: structure in their basins of attraction in the sense that the basin
292: sizes are similar. It is worth mentioning that in other aspects, like
293: orbit length or transient time\footnote{The transient time is the time
294: it takes before a trajectory enters the stable cycle.}, the basins of
295: attraction can still be very different in both models.
296:
297: From Fig.~\ref{fig:basin_size_largeK} it also can be seen that the
298: largest and next largest basins occupy more that $90\%$ of the whole
299: state space. Therefore, to a good approximation it can be assumed that
300: the dynamics takes place mainly in these two largest basins.
301:
302: %
303: %
304: %-----------------------FIGURE V ----------------------------------
305: \begin{figure}[h]
306: \psfig{file=basin_size_largeK.eps,width=3.5in,clip=} \caption{
307: \small Distribution $W(n)$ of basin-sizes
308: for: (a) the one-dimensional Kauffman lattice and (b) the Kauffman
309: net, both with $N=20$ and $K=5$. For the Kauffman net, the connections
310: between spins are chosen randomly, whereas for the lattice every spin
311: is connected to itself and to its $4$ nearest neighbors (periodic
312: boundary conditions were used). The number $n$ in the horizontal axis
313: corresponds to the $n$-th largest basin in the model.}
314: \label{fig:basin_size_largeK}
315: \end{figure}
316: %-----------------------FIGURE ^ -----------------------------------
317: %
318: %
319:
320:
321:
322: %
323: %
324: %-----------------------FIGURE V ----------------------------------
325: \begin{figure}[h]
326: \psfig{file=averagetime_largeK.eps,width=4.5in,clip=}
327: \caption{\small Average crossing time $\tau$ for different
328: Kauffman models with $N=20$ and connectivity $K=5$ (chaotic phase)
329: as a function of (a)~the noise intensity $r$ and (b)~the inverse of
330: the noise intensity. The symbols are as follows. $1$-dimensional Kauffman
331: lattice: ($\Box$) $\tau_d$ and ($\circ$) $\tau_s$. Kauffman net: ($+$)
332: $\tau_d$ and ($\ast$) $\tau_s$. Each point is the average over 4000
333: realizations of the model.}
334: \label{fig:averagetime_largeK}
335: \end{figure}
336: %-----------------------FIGURE ^ ----------------------------------
337: %
338: %
339:
340: Fig.~\ref{fig:averagetime_largeK} shows the average crossing times
341: $\tau_s$ and $\tau_d$ as functions of $r$ for the Kauffman net and the
342: $1$-dimensional lattice both with $N=20$ and $K=5$ (chaotic
343: phase). Notice that these two kinds of $N$-$K$ models exhibit very
344: similar behavior under the influence of noise.
345:
346: %=======================================================
347: \subsection{Kauffman models with small $K$}
348: %=======================================================
349:
350: The ordered phase is characterized by $K=1$ and $K=2$. In this
351: section we will present the results for the minimum value of $K$,
352: namely, $K=1$. In Fig.~\ref{fig:basin_size_smallK} we show $W(n)$
353: for a Kauffman net and a 1-dimensional Kauffman lattice, both
354: with $N=20$ and $K=1$. The connections in the Kauffman net were, as
355: usual, chosen randomly, whereas in the 1-dimensional lattice the
356: node $S_i$ was connected either to $S_{i-1}$ or to $S_{i+1}$ with
357: equal probability (we use periodic boundary conditions).
358:
359: From Fig.~\ref{fig:basin_size_smallK} it is apparent that in this
360: case, the basin structures of the Kauffman net and the Kauffman lattice are
361: less similar than in the chaotic phase. For the lattice, the two largest
362: basins of attraction no longer occupy more than 90\% of the whole state
363: space, whereas in the Kauffman net they still do. However, the response
364: to the influence of noise is mostly the same in both models, as can be seen
365: from Fig.~\ref{fig:averagetime_smallK} where the crossing times $\tau_s$
366: and $\tau_d$ are plotted as functions of the noise intensity $r$.
367:
368:
369: %
370: %
371: %-----------------------FIGURE V ----------------------------------
372: \begin{figure}[h]
373: \psfig{file=basin_size_smallK.eps,width=4in,clip=}
374: \caption{
375: \small
376: Distribution $W(n)$ of basin-sizes for: (a) the one-dimensional Kauffman
377: lattice and (b) the Kauffman net, both with $N=20$ and $K=1$(frozen
378: phase). The number $n$ in the horizontal axis has the same meaning as
379: in Fig.~\ref{fig:basin_size_largeK}.}
380: \label{fig:basin_size_smallK}
381: \end{figure}
382: %-----------------------FIGURE ^ -----------------------------------
383: %
384: %
385:
386: %
387: %
388: %-----------------------FIGURE V ----------------------------------
389: \begin{figure}[h]
390: \psfig{file=averagetime_smallK.eps,width=4.5in,clip=}
391: \caption{\small Average crossing time $\tau$ for different
392: Kauffman models with $N=20$ and connectivity $K=1$ (frozen phase) as a
393: function of (a)~the noise intensity $r$ and (b)~the inverse of the noise
394: intensity. The symbols are as follows. $1$-dimensional Kauffman
395: lattice: ($\Box$) $\tau_d$ and ($\circ$) $\tau_s$. Kauffman net: ($+$)
396: $\tau_d$ and ($\ast$) $\tau_s$. Each point is the average over 4000
397: realizations of the model.}
398: \label{fig:averagetime_smallK}
399: \end{figure}
400: %-----------------------FIGURE ^ ----------------------------------
401: %
402: %
403:
404:
405: %================================================
406: \subsection{Robust behavior of the crossing time}
407: %================================================
408:
409: Figures \ref{fig:averagetime_largeK} and \ref{fig:averagetime_smallK}
410: show that the behavior of the different Kauffman models under the
411: influence of noise have the following general characteristics, both in
412: the frozen and in the chaotic phases:
413:
414: \begin{itemize}
415: \item For small $r$, $\tau_d$ decreases as $1/r$ while $\tau_s$ is
416: nearly constant.
417: \item For large $r$, both $\tau_d$ and $\tau_s$ increase with $r$ and
418: become equal at $r=0.5$.
419: \item For intermediate values of $r$, $\tau_d$ has a minimum when
420: $\tau_s\approx\tau_d$.
421: \end{itemize}
422:
423: Let us analyze separately each one of the above characteristics.
424:
425: In the limit $r\rightarrow0$, $\tau_s$ approaches a finite value
426: $\tau_s(0)$, the mean crossing time for two trajectories in the
427: largest basin of attraction in the absence of noise. This crossing
428: time is roughly one half the average cycle length, plus one half the
429: average transient time. The transient time and the cycle length are
430: of the same order of magnitude, therefore $\tau_s(0)$ is expected to
431: be approximately equal to the mean cycle length $\langle L \rangle$ of
432: the largest basin of attraction (see Fig.~\ref{fig:intercept}a).
433:
434: On the other hand, $\tau_d$ diverges as $r\rightarrow0$. The
435: numerical data (see figures~\ref{fig:averagetime_largeK}b and
436: \ref{fig:averagetime_smallK}b) suggest that in this limit, $\tau_d$ has
437: the form
438:
439: %
440: \begin{equation}
441: \tau_d\approx {a(K,N)\over r}+b(K,N), \ \ \ \ r\rightarrow0.
442: \label{eq:1/r}
443: \end{equation}
444: %
445: The above divergence is due to the fact that, in the absence of
446: noise, there is a zero probability for a trajectory to jump
447: between different attractors. Under the deterministic dynamics,
448: every trajectory will remain within its own basin of attraction
449: for ever. In the next section we will see that the $1/r$ behavior
450: of $\tau_d$ is a consequence of the fact that the dynamics is
451: governed by one-spin flip events when $r$ is small. For large
452: values of $K$, the largest basin occupies almost the whole state
453: space. Under these circumstances, every time a one-spin flip
454: occurs, the trajectory in the next largest basin will have a
455: finite probability of diverging very substantially from the path
456: it would have followed in the absence of noise. That divergence
457: will usually force the trajectory into the largest basin. In fact,
458: for a fraction of order one of the noise events in the smaller
459: basin, the noise will flip the trajectory into the largest one.
460: Once the two trajectories are in the largest basin, they have a
461: lifetime $b(K,N)$ before they cross. This lifetime is expected to
462: be of order one of $\tau_s(0)$, the typical length of the largest
463: basin of attraction. The above can actually be seen in
464: Fig.~\ref{fig:intercept}, from which it is apparent that for large
465: $K$, $b(K,N) \approx \tau_s(0) \approx \langle L \rangle$.
466:
467: %
468: %
469: %-----------------------FIGURE V ----------------------------------
470: \begin{figure}[h]
471: \centering \psfig{file=intercept.eps,width=4.5in,clip=}
472: \caption{\small (a) Plot of the mean cycle length $\langle L \rangle$
473: as a function of $\tau_s(0)$ for a Kauffman net with $N=12$. Each
474: point corresponds to a different value of $K$, starting with $K=1$ for
475: the first point in the lower left corner of the graph and ending with
476: $K=12$ for the last point in the upper right corner. The dashed line
477: is the best linear fit to the numerical data (circles). (b) Same type
478: of graph as before but now showing the dependence of $b(K,N)$ on
479: $\tau_s(0)$. The slopes of the dashed lines are (a) $\sim 0.96$ and
480: (b) $\sim 0.90$, which shows that
481: $b(K,N)\approx\tau_s(0)\approx\langle L\rangle$. }
482: \label{fig:intercept}
483: \end{figure}
484: %-----------------------FIGURE ^ -----------------------------------
485: %
486: %
487:
488: In the opposite limit $r\rightarrow0.5$, the crossing times $\tau_s$
489: and $\tau_d$ become equal, which means that for high levels of noise,
490: the barriers between different attractors become small. When $r$
491: reaches its maximum value $0.5$, all barriers vanish. In this case,
492: both trajectories randomly jump from one state to another throughout
493: the state space, and both $\tau_s$ and $\tau_d$ become equal to the
494: time it takes for two random walks to cross. As derived in
495: section~\ref{subsec:large_r}, this crossing time is the solution to
496: the ``birthday problem'', i.e.
497:
498: %
499: \begin{equation}
500: \tau_s = \tau_d\propto 2^{N/2}
501: \label{eq:maximum_r}
502: \end{equation}
503: %
504:
505: In this way we have obtained a qualitative description of the
506: limiting cases of figures~\ref{fig:averagetime_largeK} and
507: \ref{fig:averagetime_smallK}. The one
508: qualitative feature left to describe is the crossover from the small
509: $r$ to the large $r$ behavior. As one can see from these figures, the
510: crossover occurs when the two times $\tau_s$ and $\tau_d$ become
511: roughly equal. This in turn happens when
512: %
513: \begin{equation}
514: r \sim a(K,N)/ \tau_s(0) \sim 1/N
515: \label{eq:interim_r}
516: \end{equation}
517: %
518: Thus, the minimum in $\tau_d$ occurring between the two previous limit
519: values of $r$, can be interpreted as the result of a sort of
520: ``competition'' between the randomness in the system (coming from the
521: presence of noise), and the barriers separating the attractors (which
522: come from the deterministic dynamics of the system).
523:
524: %===================================================
525: \subsection{A Kauffman model with equal basin-sizes}
526: %===================================================
527:
528: Finally, we would like to mention that the above results are also true
529: for Kauffman models in which all the basins of attraction have the
530: same weight. As an example, consider a Kauffman lattice with $N=20$
531: and $K=1$ in which every spin is connected to itself. For $K=1$ there
532: are only four Boolean functions $f_i(S)$: tautology $f_i(S)=1$,
533: contradiction $f_i(S)=0$, identity $f_i(S)=S$ and negation
534: $f_i(S)=1-S$. Imagine then the very specific realization in which two
535: of the coupling functions are identity, two are negation and all the
536: others are either tautology or contradiction. By simple analysis, we
537: know that for this specific model, the whole state space is composed
538: of eight basins of attraction with equal
539: size. Fig.~\ref{fig:averagetime_random} shows the crossing time $\tau$
540: as a function of $r$ for this particular model. Since in this case all
541: the basins of attraction have the same size, the two initial
542: conditions needed to compute $\tau$ were chosen randomly among the
543: whole state space. Again, the $\tau\sim1/r$ behavior for small $r$ and
544: the $\tau\propto 2^{N/2}$ behavior for $r\rightarrow0.5$ are obtained.
545: Of course we can construct many other models by choosing different
546: coupling functions $f_i$ for this $K=1$ self-correlated case. All the
547: Kauffman models we have explored have shown this kind of behavior
548: under noise.
549:
550:
551: %
552: %
553: %-----------------------FIGURE V ----------------------------------
554: \begin{figure}[h]
555: \psfig{file=averagetime_random.eps,width=4.5in,clip=}
556: \caption{\small Average crossing time $\tau$($\Diamond$) for a
557: Kauffman lattice with $N=20$ and connectivity $K=1$ as a function
558: of (a)~the noise intensity $r$ and (b)~the inverse of the noise
559: intensity. For this model, every spin is correlated to itself and
560: we choose the Boolean functions in such a way that the whole state
561: space is composed of $8$ basins of attraction with equal size.
562: Each point is the average over $10000$ realizations of the model.}
563: \label{fig:averagetime_random}
564: \end{figure}
565: %-----------------------FIGURE ^ ----------------------------------
566: %
567: %
568:
569:
570: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
571: \section{Theoretical analysis}\label{sec:theoretical}
572:
573: %=======================================================
574: \subsection{Small $r$ limit} \label{subsec:small_r}
575: %=======================================================
576:
577: In this region, the main characteristic of average crossing time is
578: that $\tau_d\sim 1/r$. The reason for this dependence is that the
579: stochastic dynamics is dominated by one-spin flip events, in the
580: following sense. The probability of a one-spin flip event ($\sim r$)
581: is much larger than the probability of a two-spin flip event ($\sim
582: r^2$). If the probability to jump to a different basin of attraction
583: in a one-spin flip event is significantly different from zero, then
584: the dynamics will be dominated only by this kind of events. Even if it
585: was necessary to flip two spins to jump from one basin to another,
586: this process can be decomposed into two one-spin flip events occurring
587: sequentially, instead of being carried out at once in one two-spin
588: flip event.
589:
590: As we have shown in section~\ref{sec:numerical}, the $1/r$ behavior is
591: present in a wide variety of Kauffman models. This in turn, implies
592: that the one-spin flip events dominate the dynamics for small values
593: of the noise. In this subsection, our goal is to derive an expression
594: for the coefficient $a(K,N)$ for Kauffman nets. To do so, we will make
595: the assumption that the dynamics takes place only in the two largest
596: basins of attraction. Although the $\tau_d\sim 1/r$ behavior is
597: generally true, as we have found, the preceding assumption is not true
598: for all Kauffman models, especially for those with small values of
599: $K$, but as we show below, it becomes more valid as $K$ increases.
600:
601: To start the calculation of $a(K,N)$, let $P_{1,2}$ be the probability
602: for jumping from the largest basin to the next largest basin with a
603: one-spin flip event and $P_{2,1}$ be a similar probability but jumping
604: in the opposite direction\footnote{By definition, the ratio of
605: $P_{1,2}$ and $P_{2,1}$ is strictly the inverse of the ratio of the
606: size of the largest basin to the next largest one in one
607: realization.}. Let us also define $Q_1$ and $Q_2$ as the
608: probabilities of remaining in the largest basin and in the next
609: largest one, respectively, after one-spin flip event. Simulations show
610: that for Kauffman nets these probabilities have a slight dependence on
611: $N$ but a very strong dependence on $K$. The result is that
612: $P_{1,2}+Q_1\approx P_{2,1}+Q_{2}\approx M$, where $M$ is
613: approximately constant for all values of $K$ and $M>0.92$ (see
614: Fig.~\ref{fig:probabilities}).
615:
616: However, the dynamics depends not only on the total sum $M$ but
617: also on the particular values of $P_{1,2}$, $Q_1$, $P_{2,1}$ and
618: $Q_2$. For small values of $K$, both $P_{1,2}$ and $P_{2,1}$ are
619: very small and comparable with $(1-M)$. So, even though $M$ is
620: large, the interaction between the two largest basins is weak in
621: the sense that the probability of jumping into smaller basins is
622: of the same order as $P_{1,2}$ and $P_{2,1}$. Therefore, for
623: small values of $K$ the smaller basins play a significant role in
624: the dynamics of the system. The above can be seen in
625: Fig.~\ref{fig:probabilities}, in which the probabilities
626: $P_{1,2}$, $P_{2,1}$, $Q_1$, $Q_2$, and the sum $M$ are plotted as
627: functions of $K$. From this figure it is apparent that the two
628: largest basins of attraction are the dominant ones for large
629: values of $K$ (say $K \geq 5$). The closeness of $M$ to $1$ means
630: that a trajectory will seldom jump into a basin other than the two
631: largest ones with only one-spin flip event. In view of this
632: result, in some of the arguments below we will assume that $K$ is
633: sufficiently large so that the dynamics takes place only in the
634: two largest basins of attraction.
635:
636: %
637: %
638: %-----------------------FIGURE V ----------------------------------
639: \begin{figure}[h]
640: \centering \psfig{file=probabilities.eps,width=4 in,clip=}
641: \caption{\small Plot of the probabilities $P_{1,2}$ ($\Diamond$),
642: $P_{2,1}$ ($\triangle$), $Q_1$ ($\Box$) and $Q_2$ ($\times$) as
643: functions of $K$, for a Kauffman net with $N=12$. Also shown is
644: $M$ as a function of $K$, obtained as $M=P_{1,2}+Q_1$ ($\ast$),
645: and as $M=P_{2,1}+Q_2$ ($\circ$). Note that even though $M$ is a
646: constant for all values of $K$, the probabilities $P_{1,2}$ and
647: $P_{2,1}$ are rather small for small values of $K$. }
648: \label{fig:probabilities}
649: \end{figure}
650: %-----------------------FIGURE ^ -----------------------------------
651: %
652: %
653:
654: With the information about these probabilities, we can give an
655: approximate calculation of $\tau_d$ as a function of $r$ for the
656: case in which $K$ is large. We know that every spin violates the
657: deterministic rule (\ref{eq:definition}) with probability $r$.
658: Therefore, the probability of a one-spin flip event is $Nr$ and
659: consequently the expected time for this event to occur is
660: $T={1/(Nr)}$. This is true for all configurations. For
661: sufficiently small values of $r$, this expectation time is much
662: longer than the average crossing time for two configurations in
663: the same basin. Hence, once two configurations jump into the same
664: basin, their trajectories meet before the next spin-flip event
665: becomes possible.
666:
667: There are two cases in which the two trajectories meet after the
668: occurrence of a one-spin flip event at time $T$: the configuration in
669: the largest basin remains in it while the configuration in the next
670: largest basin jumps into the largest one, or vice versa. The above
671: occurs with probability $(Q_{1}P_{2,1}+Q_{2}P_{1,2})$. Similarly we
672: get the probability for the crossing of the two trajectories after
673: one-spin flip events at $2T$, $3T$, etc. The average value of $\tau_d$
674: is then:
675: %
676: \begin{eqnarray}
677: \tau_d & \approx & T\cdot (Q_{1}P_{2,1}+Q_{2}P_{1,2})+2T\cdot
678: (Q_{1}Q_{2}+P_{1,2}P_{2,1})\cdot (Q_{1}P_{2,1}+Q_{2}P_{1,2})+\cdots
679: \nonumber \\
680: &=&\sum_{m=1}^{\infty} m\cdot T\cdot
681: (Q_{1}P_{2,1}+Q_{2}P_{1,2})\cdot
682: (Q_{1}Q_{2}+P_{1,2}P_{2,1})^{m-1}\nonumber \\
683: &=&{1\over Nr}\cdot {Q_{1}P_{2,1}+Q_{2}P_{1,2}\over
684: (1-Q_{1}Q_{2}-P_{1,2}P_{2,1})^2}
685: \nonumber \\
686: &=& \frac{a(K,N)}{r}
687: \label{eq:coefficient}
688: \end{eqnarray}
689: %
690: where $a(K,N)$ is explicitly given by
691: %
692: \begin{equation}
693: a(K,N)=\frac{1}{N}\frac{Q_{1}P_{2,1}+Q_{2}P_{1,2}}
694: {(1-Q_{1}Q_{2}-P_{1,2}P_{2,1})^2}
695: \label{eq:coefficient1}
696: \end{equation}
697: %
698: In the above derivation, which is true for the case in which $K$
699: is large, we have ignored the time $b(K,N)$ it takes for two
700: trajectories to cross after they have jumped into the same basin.
701: Fig.~\ref{fig:coefficient} shows the coefficient $a(K,N)$ obtained
702: from equation~(\ref{eq:coefficient1}) and from simulation, for
703: different values of $K$ in a Kauffman net with $N=12$. It can be
704: seen that the simulation and the theoretical result agree very
705: well for $K\geq 5$. It is worth emphasizing that for small values
706: of $K$ and other Kauffman models where the two largest basins
707: don't have such dominance, the effect of other basins besides the
708: largest and next largest ones has to be considered to perform an
709: accurate derivation for the coefficient $a(K,N)$.
710:
711: %
712: %
713: %-----------------------FIGURE V ----------------------------------
714: \begin{figure}[h]
715: \psfig{file=coefficient.eps,width=4.5in,clip=} \caption{The
716: coefficient $a(K,N)$ in Eqn.~(\ref{eq:1/r}) for different values of
717: $K$ for a Kauffman net with $N=12$. ($\times$) is the result obtained
718: from simulations by sampling the whole state space. ($\Box$) is the
719: result obtained from the theoretical result
720: Eqn.~(\ref{eq:coefficient1}).} \label{fig:coefficient}
721: \end{figure}
722: %-----------------------FIGURE ^ -----------------------------------
723: %
724: %
725:
726:
727:
728:
729: %==========================================================
730: \subsection{Large $r$ limit} \label{subsec:large_r}
731: %==========================================================
732:
733:
734: When $r$ acquires its maximum value $0.5$, the barriers between
735: different attractors vanish and the dynamics transforms into
736: a random mapping of the state space into itself
737: \cite{60.1,87.9}. In this limit, the crossing times $\tau_s$
738: and $\tau_d$ become indistinguishable and we will refer to both of
739: them simply as $\tau$. To understand the limit $r\rightarrow0.5$ we
740: will first give a simple ``birthday-problem'' argument to obtain the
741: order of magnitude of $\tau$. Then we will proceed to a more elaborate
742: analysis to obtain an approximate expression for $\tau$ as a function
743: of $r$, valid for $r$ close to $0.5$.
744:
745: Imagine two walkers moving at random through a space of dimension
746: $\Omega=2^N$. They go through a number of steps $n=1, 2, 3\dots$ As
747: each step is completed, they have a total number of chances $C_n$ on
748: landing on some place previously covered by the other walker. At the
749: first step we could have the two walkers at identical positions. Thus,
750: $C_1=1$. At the second step, $C_2=3+C_1$ since each can land on the
751: original position of the other or they can both land at precisely the
752: same place. After $n$ steps $C_n=C_{n-1} +2n+1= n^2$. For large
753: $\Omega$ and small $n$, the probability of having collided with the
754: path is then of order $p_n \approx C_n/ \Omega$. The average time for
755: crossing is then roughly given by the $n$-value for which $p_n$
756: becomes of order unity, so that $n^2$ is of order $\Omega$ or $n =
757: O(\Omega^{1/2})$. Therefore, the crossing time $\tau$ satisfies
758: $\tau\propto\Omega^{1/2}$.
759:
760: To derive a more precise functional relation between $\tau$ and $r$ we
761: have to compute the probability $p_c(t)$ for two trajectories
762: $\{\Sigma_0,\Sigma_1,\dots, \Sigma_\tau\}$ and
763: $\{\tilde{\Sigma}_0,\tilde{\Sigma}_1,\dots, \tilde{\Sigma}_\tau\}$ to
764: cross at time $\tau$. In order to do that, we have to have first the
765: probability $p$ for two configurations $\Sigma_t$ and
766: $\tilde{\Sigma}_{\tilde{t}}$ to be the same (note that $t$ and
767: $\tilde{t}$ might be different). Let $S_i(t-1)$ be in $\Sigma_{t-1}$
768: and $\tilde{S}_i(\tilde{t}-1)$ be the corresponding spin in
769: $\tilde{\Sigma}_{\tilde{t}-1}$. Since these spins are in the same
770: position (each in its respective configuration), the deterministic
771: rule $f_i$ they obey is the same. Notice that the dynamical equation
772: (\ref{eq:noisedefinition}) can be written as
773: %
774: \begin{equation}
775: S_i(t+1)=\left\{
776: \begin{array}{ll}
777: f_i&\mbox{with probability $1-2r$},\\
778: \mbox{evolve randomly}&\mbox{with probability $2r$}.
779: \end{array}
780: \right. \label{eq:noisedefinition_1}
781: \end{equation}
782: %
783: From this expression it follows that the probability for $S_i(t-1)$
784: and $\tilde{S}_i(\tilde{t}-1)$ to evolve according to the
785: deterministic rule $f_i$ is $(1-2r)^2$. Let us denote by $p_1$ the
786: probability that $S_i(t)=\tilde{S}_i(\tilde{t})$ when both spins are
787: updated according to the deterministic rule $f_i$. To calculate $p_1$
788: we follow the annealed approximation introduced by Derrida and Pomeau
789: \cite{86.3}, which leads us to the following two possibilities:
790:
791: \begin{enumerate}
792: \item
793: The $K$ inputs of $S_i(t-1)$ and $\tilde{S}_i(\tilde{t}-1)$ are the
794: same, which occurs with probability $1/2^K$. When this happens,
795: $S_i(t)=\tilde{S}_i(\tilde{t})$ with probability $1$.
796: \item
797: At least one of the inputs is different, which occurs with probability
798: $(1-1/2^K)$. In this case, if the evolution rules $f_i$ are assigned
799: in a sufficiently random way, there is a probability of 1/2 that
800: $S_i(t)=\tilde{S}_i(\tilde{t})$.
801: \end{enumerate}
802: %
803: From the above it follows that
804: %
805: \[
806: p_1=(1-2r)^2\left[\frac{1}{2^K}+\left(1-\frac{1}{2^K}\right)\cdot
807: \frac{1}{2}\right]
808: \]
809: %
810: On the other hand, the probability $p_2$ that $S_i(t)=\tilde{S}_i(\tilde{t})$
811: when the evolution rule $f_i$ is violated in one or both of the
812: configurations, is simply given by
813: %
814: \[
815: p_2=[1-(1-2r)^2]\cdot {1\over 2}
816: \]
817: %
818: Combining the values of $p_1$ and $p_2$ given above, the probability
819: $p$ for both configurations $\Sigma_{t}$ and $\tilde{\Sigma}_{\tilde{t}}$
820: to be equal is
821: %
822: \begin{eqnarray}
823: p&=&\left \{ (1-2r)^2\left [{1\over 2^K}+(1-{1\over 2^K})\cdot
824: {1\over 2}\right ]+[1-(1-2r)^2]\cdot {1\over 2}\right \} ^N\nonumber \\
825: &=&{1\over 2^N}\cdot \left \{ 1+{1\over 2^K}\cdot (1-2r)^2\right
826: \}^N \label{eq:p}
827: \end{eqnarray}
828: %
829: If $q(t)$ is the probability that the two trajectories have not yet
830: crossed at time $t$, then $p_c(t)$, the probability for the two
831: trajectories to cross at time $t$, is given by
832: %
833: \begin{equation}
834: p_c(t)=q(t-1)-q(t) \label{eq:p_c}
835: \end{equation}
836: %
837: The two trajectories are still separated at time $t$ if none of
838: the configurations $\{\Sigma_0,\Sigma_1,\dots,\Sigma_t\}$ is equal
839: to any of the configurations
840: $\{\tilde{\Sigma}_0,\tilde{\Sigma}_1,\dots,\tilde{\Sigma}_{t}\}$.
841: The probability for this to happen is
842: \[
843: q(t)=(1-p)^{(t+1)^2}
844: \]
845: %
846: Substituting this value of $q(t)$ into equation (\ref{eq:p_c}) we
847: get
848: \begin{equation}
849: p_c(t)= (1-p)^{t^2}-(1-p)^{(t+1)^2}
850: \end{equation}
851: %
852: Therefore, the average crossing time $\tau=\sum_{t=1}^{\infty}t\cdot
853: p_c(t)$ is given by
854: %
855: \begin{eqnarray}
856: \tau&=&\sum_{t=1}^{+\infty}t\cdot[(1-p)^{t^2}-(1-p)^{(t+1)^2}]\\
857: &\approx& \int_0^{+\infty}-t{d(1-p)^{t^2}\over dt}dt \nonumber \\
858: &=&{1\over 2}\sqrt{\pi\over -\ln{(1-p)}} \nonumber
859: \label{theory1}
860: \end{eqnarray}
861: %
862: Expanding the logarithm in the above equation around $p=0$, and
863: retaining only the terms up to the the first order, we finally get
864: %
865: \begin{equation}
866: \tau \approx {\sqrt{\pi} \over 2}\left [ 2\over 1+{1\over
867: 2^K}\cdot (1-2r)^2 \right ]^{N/2} \label{eq:tau}
868: \end{equation}
869:
870: It can be seen that Eqn.~\ref{eq:tau} is consistent with the
871: ``birthday-problem'' argument for the case $r=0.5$. When $r$ is not
872: exactly $0.5$, we do not have a simple birthday problem because the
873: coupling between different elements, and consequently the functions
874: $f_i$, still play a role in the dynamics. However, it is clear from
875: the above equation that the problem can be viewed as a birthday
876: problem with an effective state space $\Omega_{eff}=\left [ 2\over
877: 1+(1-2r)^2/2^K \right ]^{N}$.
878:
879: %
880: %
881: %-----------------------FIGURE V ---------------------------------
882: \begin{figure}[h]
883: \psfig{file=theory_t.eps,width=4.5in,clip=} \caption{ Average
884: crossing time $\tau$ as a function of $r$ for two Kauffman nets with
885: $N=20$ and connectivities (a) $K=1$ and (b) $K=5$. In both graphs the
886: dashed curve is the result of the numerical simulation for $\tau_d$,
887: whereas the solid line is the plot of the analytic expression
888: (\ref{theory1}).} \label{fig:theory_t}
889: \end{figure}
890: %-----------------------FIGURE ^ ----------------------------------
891: %
892: %
893:
894: Fig.~\ref{fig:theory_t} compares the theoretical result
895: (\ref{eq:tau}) with the numerical simulation for $K=1$ and $K=5$.
896: As can be seen, the analytic result approximates very well the
897: numerical data in the region of $r$ close to $0.5$. In this
898: region the annealed approximation holds because the noise
899: breaks the correlations between the spins. However, for small
900: values of the noise those correlations are important and cannot
901: be neglected. We therefore do not expect agreement between the
902: numerical and theoretical results for small values of $r$ since
903: in this region the annealed appriximation is not longer valid.
904:
905:
906: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
907: \section{Conclusions}\label{sec:conclusions}
908:
909: We have considered the effect of external perturbations (noise) in the
910: dynamics of the Kauffman model. The behavior of both, the Kauffman net
911: and the Kauffman lattice under the influence of noise is very similar,
912: even though these models might have a quite different structure in
913: their basins of attraction. In this sense, the response of the
914: Kauffman models to the effect of noise can be considered as a very
915: robust property.
916:
917: In the limit $r\rightarrow0$, the most important property is the $1/r$
918: behavior of the crossing time $\tau_d$, which has been always present
919: in the Kauffman models we have studied so far. This $1/r$ behavior is
920: a consequence of the fact that, for small values of the noise, the
921: dynamics is dominated by one-spin flip events. An approximate equation
922: relating $\tau_d$ and $r$ was obtained by taking into account the fact
923: that, for large values of the connectivity, most of the dynamics takes
924: place in the largest and next largest basins of attraction.
925:
926: In the second limit $r\rightarrow0.5$, the barriers between different
927: attractors disappear and the dynamics transforms into a random mapping
928: of the state space into itself. As a consequence, $\tau_s$ and
929: $\tau_d$ become equal. In the case in which $r=0.5$, the crossing time
930: between the two trajectories can be seen as the solution of a
931: ``birthday problem'' in a space of size $\Omega=2^N$. For other values
932: of $r$, but still close to $0.5$, the correlations between spins have
933: to be taken into account, which have the effect of reducing the size
934: of the region of the state space explored by the dynamics.
935:
936: Between these two limit cases for the noise, there is a minimum in the
937: value of $\tau_d$ as a function of $r$. In a loose sense, this minimum
938: could be interpreted as the result of a ``competition'' between the
939: randomness generated by the noise, which tends to homogenize the state
940: space by diminishing the barriers across different attractors, and the
941: deterministic dynamics, which tends to confine two trajectories within
942: the same basin. To analyze this region it would be necessary to
943: consider multiple-spin flip events as well as long-time step
944: correlations.
945:
946: The results and techniques presented in this work could be extended to
947: other systems acting under the influence of noise in order to provide
948: them with a robust characterization.
949:
950: \subsection*{Acknowledgements}
951: This work was supported in part by the MRSEC Program of the National
952: Science Foundation under award number 9808595, and by the NSF DMR
953: 0094569. We also thank to the Santa Fe Institute of Complex Systems
954: for partial support through the David and Lucile Packard Foundation
955: Program in the Study of Robustness.
956:
957: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
958:
959: \pagebreak
960:
961: \bibliographystyle{plain}
962: \bibliography{bibliography}
963:
964: \end{document}
965: