1: \documentclass[12pt]{iopart}
2: \newcommand{\gguide}{{\it Preparing graphics for IOP journals}}
3: %Uncomment next line if AMS fonts required
4: %\usepackage{iopams}
5: \usepackage{graphicx}
6:
7: \begin{document}
8:
9: \title[Transient state dynamics]
10: {Neural networks with transient state dynamics}
11:
12: \author{Claudius Gros}
13:
14: \address{Institute of Theoretical Physics
15: J.W.\ Goethe University Frankfurt,
16: 60438 Frankfurt, Germany}
17: \ead{gros07--@--itp.uni-frankfurt.de}
18: \begin{abstract}
19: We investigate dynamical systems characterized by a time series
20: of distinct semi-stable activity patterns, as they are
21: observed in cortical neural activity patterns. We propose and
22: discuss a general mechanism allowing for an adiabatic
23: continuation between attractor networks and a specific
24: adjoined transient-state network, which is
25: strictly dissipative. Dynamical systems with
26: transient states retain functionality when their working point
27: is autoregulated - avoiding prolonged periods of stasis or
28: drifting into a regime of rapid fluctuations. We show, within
29: a continuous-time neural network model, that a single local updating rule
30: for online learning allows simultaneously (a) for information storage
31: via unsupervised Hebbian-type learning (b) for adaptive regulation of the
32: working point and (c) for the suppression of runaway synaptic growth.
33: Simulation results are presented, the spontaneous breaking of
34: time-reversal symmetry and link symmetry are discussed.
35: \end{abstract}
36:
37: %Uncomment for PACS numbers title message
38: %\pacs{00.00, 20.00, 42.10}
39: % Keywords required only for MST, PB, PMB, PM, JOA, JOB?
40: %\vspace{2pc}
41: %\noindent{\it Keywords}: Article preparation, IOP journals
42: % Uncomment for Submitted to journal title message
43: %\submitto{\JPA}
44: % Comment out if separate title page not required
45: \maketitle
46: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
47: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
48:
49: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
50: \section{Introduction}
51: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
52:
53: Dynamical systems are often classified with respect to
54: their long-time behaviors, which might be, e.g., chaotic or
55: regular \cite{katok95}. Of special interest are attractors,
56: cycles and limiting cycles, as they determine the fate of all
57: orbits starting within their respective basins of attraction.
58:
59: Attractor states play a central role in the theory
60: of recurrent neural networks, serving the role of memories
61: with the capability to generalize and to reconstruct
62: a complete memory from partial initial information \cite{hopfield82}.
63: Attractor states in recurrent neural networks
64: face however a fundamental functional
65: dichotomy, whenever the network is considered as a functional
66: subunit of an encompassing autonomous information
67: processing system, {\it viz} an autonomous cognitive system
68: \cite{gros07}.
69: The information processing comes essentially to a standstill
70: once the trajectory closes in at one of the attractors. Restarting
71: the system `by hand' is a viable option for technical
72: applications of neural networks, but not within the
73: context of autonomously operating cognitive systems.
74:
75: One obvious way out of this dilemma would be to consider
76: only dynamical systems without attractor states, i.e.\
77: with a kind of continously ongoing `fluctuating dynamics',
78: as illustrated in Fig.\ \ref{fig_transStates}, which
79: might possibly be chaotic in the strict sense of
80: dynamical system theory. The problem is
81: then, however, the decision-making process. Without well
82: defined states, which last for certain minimal periods, the
83: system has no definite information-carrying states onto which
84: it could base the generation of its
85: output signals. It is interesting to note in this context,
86: that indications for quasi-stationary patterns in cortical
87: neural activity have been observed \cite{abeles95,kenet03,ringach03}.
88: These quasi-stationary states can be
89: analyzed using multivariate time-series analysis,
90: indicating self-organized patterns of brain activity
91: \cite{hutt03}. Interestingly, studies of EEG recordings
92: have been interpreted in terms of brain states showing
93: aperiodic evolution states going through sequences of
94: attractors that on access support the experience of
95: remembering \cite{freeman03}.
96: These findings suggest that `transient state dynamics',
97: as illustrated in Fig.\ \ref{fig_transStates}, might
98: be of importance for cortical firing patterns.
99:
100: It is possible, from the viewpoint of dynamical
101: system theory, to consider transient states as
102: well defined periods when the orbit
103: approaches an attractor ruin. With a
104: transient attractor, or attractor ruin,
105: we denote here a point in phase
106: space which could be turned
107: continously into a stable attractor when
108: tuning certain of the parameters entering the
109: evolution equations of the dynamical system.
110: The dynamics slows down close to the attractor ruin
111: and well defined transient states emerge within the ensemble
112: of dynamical variables.
113: The notion of transient state dynamics is related
114: conceptually to chaotic itinerancy
115: \cite{kaneko03}, a term used to characterize
116: dynamical systems for which
117: chaotic high-dimensional orbits stay
118: intermittently close to low-dimensional attractor
119: ruins for certain periods. Instability due to
120: dynamic interactions or noise is necessary for
121: the appearance of chaotic itinerancy.
122:
123: Having argued that transient-state dynamics
124: might be of importance for a wide range
125: of real-world dynamical systems,
126: the question is then of how to generate
127: such kind of dynamical behavior in a
128: controllable fashion and in a manner applicable
129: to a variety of starting systems. {\it Viz}
130: we are interested in neural networks which
131: generate transient states dynamics in terms
132: of a meaningful time series of
133: states approaching arbitrarily close
134: predefined attractor ruins.
135:
136: The approach we will follow here is to start with an
137: original attractor neural network and to transform then the
138: set of stable attractors into transient attractors
139: by coupling to auxiliary local variables, which we denote
140: `reservoirs', governed by long time scales. We note,
141: that related issues have been investigated in the context of
142: discrete-time, phase coupled oscillators \cite{timme02},
143: for networks aimed at language processing in terms of
144: `latching transitions' \cite{treves05,kropff06}, and
145: in the context of `winnerless competitions'
146: \cite{rabinovich01,seliger03,rabinovich06a}.
147: Further examples of neural networks capable of generating a
148: time-series of subsequent states are
149: neural networks with time-dependent
150: asymmetric synaptic strengths \cite{sompolinsky86}
151: or dynamical thresholds \cite{horn89}. We also
152: note that the occurrence of spontaneous fluctuating dynamics
153: has been studied \cite{metzler01}, especially
154: in relation to the underlying network geometry \cite{paula06}.
155:
156: An intrinsic task of neural networks is to learn and to
157: adapt to incoming stimuli. This implies, for adaptive neural
158: networks, a continuous modification of their dynamical
159: properties. The learning process could consequently
160: take the network, if no precautions are taken,
161: out of its intended working regime, the
162: regime of transient state dynamics.
163: Here we will show
164: that it is possible to formulate local learning rules
165: which keep the system in its proper dynamical state
166: by optimizing continously its own working point. To be
167: concrete, let us denote with $\bar t$ the average
168: duration of quasi-stable transient states and with
169: $\Delta t$ the typical time needed for the transition from
170: one quasi-stationary state to the next. The dynamical
171: working point can then be defined as the ratio $\Delta t/\bar t$.
172:
173: These time scales, $\bar t$ and $\Delta t$, result,
174: for the network of cortical neurons, from the properties
175: of the individual neurons, which are essentially
176: time-independent, and from the synaptic strengths,
177: which are slow dynamical variables
178: subject to Hebbian-type learning \cite{arbib02}.
179: It then follows, that the modifications of the inter-neural
180: synaptic strengths have a dual functionality: on one
181: side they are involved in memory storage tasks \cite{arbib02},
182: and on the other side they need to retain the working point
183: in the optimal regime. Here we show that this
184: dual functionality can be achieved within
185: a generalized neural network model.
186: We show that working-point optimization is obtained
187: when the Hebbian learning-rule is reformulated
188: as an optimization procedure, resulting in
189: a competition among the set of synapses leading
190: to an individual neuron. The resulting learning-rule
191: turns out to be closely related to rules found to
192: optimize the memory-storage capacity \cite{chechik01}.
193:
194: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
195: \begin{figure}[t]
196: \centerline{
197: \includegraphics*[width=0.60\textwidth]{transStates.eps}
198: }
199: \caption{Illustration of fluctuating (top) and
200: transient state dynamics (bottom).
201: }
202: \label{fig_transStates}
203: \end{figure}
204: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
205:
206:
207: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
208: \section{Model}
209: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
210:
211: %----------------%
212: \subsection{Clique encoding}
213: %----------------%
214:
215: Neural networks with sparse coding, {\it viz} with
216: low mean firing rates, have very large memory storage
217: capacities \cite{okada96}. Sparse coding results, in
218: extremis, in a `one-winner-take-all' configuration, for which
219: a single unit encodes exactly one memory. In this limit
220: the storage capacity is, however, reduced again and linearly
221: proportional to the network size, as in the original
222: Hopfield model \cite{amit85}.
223: Here we opt for the intermediate case of `clique encoding'.
224: A clique is, in terms of graph theory, a fully
225: interconnected subgraph, as illustrated in
226: Fig.\ \ref{fig_7sites} for a 7-site network.
227: Clique encoding corresponds to a `several-winners-take-all'
228: setup. All members of the winning clique mutually excite
229: each other while suppressing the activities of all
230: out-of-clique neurons to zero.
231:
232: We note, that the number of cliques can be
233: very large. For illustration let us consider a random
234: Erd\"os--R\'enyi graph with $N$ vertices and linking
235: probability $p$. The overall number of cliques containing
236: $Z$ vertices is then statistically given by
237: %
238: \begin{equation}
239: \left( \begin{array}{c} N \\ Z \end{array} \right)
240: p^{Z(Z-1)/2}\left(1-p^Z\right)^{N-Z}~,
241: \label{cogSys_N_z}
242: \end{equation}
243: %
244: where $p^{Z(Z-1)/2}$ is the probability of having $Z$
245: sites of the graph fully interconnected by $Z(Z-1)/2$
246: edges and where the last term is the probability that
247: every single of the $N-Z$ out-of-cliques vertices is
248: not simultaneously connected to all $Z$ sites of the clique.
249:
250: Networks with clique encoding are especially well suited for
251: transient state dynamics, as we will discuss further below,
252: and are biologically plausible. Extensive sensory preprocessing
253: is known to occur in the respective cortical areas of the brain
254: \cite{arbib02}, leading to representations of features and
255: objects by individual neurons or small cell assemblies. In this
256: framework a site, {\it viz} a neural center,
257: of the effective neural network considered here
258: corresponds to such a small cell assembly and a clique to a stable
259: representation of a memory, by binding together a finite
260: set of features extracted by the preprocessing
261: algorithms from the sensory input stream.
262:
263: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
264: \begin{figure}[t]
265: \centerline{
266: \includegraphics*[width=0.40\textwidth]{7sites.eps}\hspace{1ex}
267: \includegraphics*[width=0.45\textwidth]{AI7.eps}
268: }
269: \caption{Geometry and simulation results for
270: a small, 7-site network.\newline
271: Left: The links with $w_{i,j}>0$, containing six cliques,
272: (0,1), (0,6), (3,6), (1,2,3) (which is highlighted),
273: (4,5,6) and (1,2,4,5). \newline
274: Right:
275: As a function of time,
276: the activities $x_i(t)$ (solid lines)
277: and the respective reservoirs $\varphi_i(t)$ (dashed lines)
278: for the transient state dynamics $(4,5,6)\rightarrow(1,2,3)
279: \rightarrow (0,6) \rightarrow(1,2,4,5)$. For the
280: parameters values see Sect.\ \ref{sec_strict}.
281: \label{fig_7sites}
282: }
283: \end{figure}
284: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
285:
286: %----------------%
287: \subsection{Continuous time dynamics}
288: %----------------%
289:
290: For our study of possible mechanisms of transient state
291: dynamics in the context of neural networks we consider
292: $i=1,...,N$ artificial neurons with rate encoding
293: $x_i(t)$ and continuous time $t\in [0,\infty]$. Let us
294: comment shortly on the last point. The majority of research
295: in the field of artificial neural networks deals with
296: the case of discrete time $t=0,1,2,...$ \cite{arbib02}.
297: We are however interested, as discussed in the introduction,
298: in networks exhibiting autonomously generated dynamical behaviors,
299: as they typically occur in the context of complete
300: autonomous cognitive systems. We are therefore interested
301: in networks having update rules being compatible with
302: the interaction with other components of a cognitive
303: system. Discrete time updating is not suitable in
304: this context, since the resulting dynamical characteristics (i)
305: depend on the choice of synchronous vs.\ asynchronous
306: updating and (ii) are strongly influenced when effective
307: recurrent loops arise due to the coupling to other
308: components of the autonomous cognitive system. We
309: therefore consider and study here a model with continuous
310: time.
311:
312: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
313: \begin{figure}[t]
314: \centerline{
315: \includegraphics[height=0.30\textwidth]{reservoirFunctions.eps}
316: \hspace{6ex}
317: \includegraphics[height=0.30\textwidth]{allowed.eps}
318: }
319: \caption{Left: Illustration of the reservoir functions
320: $f_{z/w}(\varphi)$, see Eq.\ \ref{cogSys_ri},
321: of sigmoidal form \cite{note_functions}
322: with respective turning points
323: $\varphi_c^{(f/z)}$, a width $\Gamma_\varphi$ and a minimal
324: value $f_z^{(min)}=0$. \newline
325: Right: Distribution of the synaptic strength for the
326: inhibitory links $z_{ij}<-|z|$ and the active
327: excitatory links $0<w_{ij}<w$ leading to
328: clique encoding. Note, that $w$ is not
329: a strict upper bound, due to the optimization
330: procedure (\ref{cogSys_w_L_dot_opt}).
331: The shaded area just below zero
332: is related to the inactive $w_{ij}$, see
333: Eqs.\ (\ref{cogSys_z_t}) and (\ref{cogSys_w_L_dot_opt}).
334: }
335: \label{cogSys_fig_gaps}
336: \end{figure}
337: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
338:
339:
340: %----------------%
341: \subsection{Neural network model}
342: %----------------%
343:
344: We denote the state variables encoding the
345: activity level by $x_i(t)$ and assume them
346: to be continuous variables, $x_i\in[0,1]$.
347: Additionally, we introduce for every site a
348: variable $\varphi_i(t)\in[0,1]$, termed `reservoir',
349: which serves as a fatigue memory facilitating
350: the self generated time series of transient states.
351: We consider the following set of differential
352: equations:
353: %
354: %
355: \begin{eqnarray} \label{cogSys_xdot}
356: \dot x_i &=& (1-x_i)\,\Theta(r_i)\,r_i \,+\, x_i\,\Theta(-r_i)\,r_i
357: \\ \label{cogSys_ri}
358: r_i &=&
359: \sum_{j=1}^N \Big[
360: f_w(\varphi_i) \Theta(w_{ij}) w_{i,j}
361: + z_{i,j}f_z(\varphi_j)
362: \Big] x_j\ \ \
363: \\ \label{cogSys_phidot}
364: \dot\varphi_i & =&
365: \Gamma_\varphi^+\, (1-\,\varphi_i)(1-x_i/x_c)
366: \Theta(x_c-x_i)
367: \,-\, \Gamma_\varphi^-\,\varphi_i\,\Theta(x_i-x_c)
368: \\ \label{cogSys_z_t}
369: z_{ij} & =& -|z|\,\Theta(-w_{ij})
370: \end{eqnarray}
371: %
372: %
373: We now discuss some properties of
374: (\ref{cogSys_xdot}-\ref{cogSys_z_t}), which are suitably
375: modified Lotka-Volterra equations:
376: \begin{itemize}
377:
378: \item \underline{Normalization}\\
379: Eqs.\ (\ref{cogSys_xdot}-\ref{cogSys_phidot}) respect the
380: normalization $x_i,\varphi_i\in[0,1]$, due to
381: the prefactors $x_i$,$(1-x_i)$, $\varphi_i$
382: and $(1-\varphi_i)$
383: in Eqs.~(\ref{cogSys_xdot}) and (\ref{cogSys_phidot}),
384: for the respective growth and depletion processes.
385: $\Theta(r)$ is the Heaviside-step function:
386: $\Theta(r<0)=0$ and $\Theta(r>0)=1$.
387:
388: \item\underline{Synaptic strength}\\
389: The synaptic strength is split into an
390: excitatory contribution $\propto w_{i,j}$
391: and an inhibitory contribution $\propto z_{i,j}$,
392: with $w_{i,j}$ being the primary variable:
393: The inhibition $z_{i,j}$ is present only
394: when the link is not excitatory (\ref{cogSys_z_t}).
395: We have used $z\equiv-1$, {\it viz} $|z|=1$
396: throughout the manuscript, which then defines the
397: inverse reference unit for the time development.
398:
399: \item\underline{Winners-take-all network}\\
400: Eqs.~(\ref{cogSys_xdot}) and (\ref{cogSys_ri}) describe,
401: in the absence of a coupling to the reservoir via
402: $f_{z/w}(\varphi)$, a competitive winners-take-all
403: neural network with clique encoding.
404: The system relaxes towards
405: the next attractor made up of a clique
406: of $Z$ sites $(p_1,\dots,p_Z)$ connected via
407: excitatory links $w_{p_i,p_j}>0$ ($i,j=1,..,Z$).
408:
409: \item\underline{Reservoir functions}\\
410: The reservoir functions $f_{z/w}(\varphi)\in[0,1]$
411: govern the interaction in between the activity levels $x_i$
412: and the reservoir levels $\varphi_i$.
413: They may be chosen as washed out step functions
414: of sigmoidal form \cite{note_functions},
415: with a suitable width $\Gamma_{\varphi}$ and
416: inflection points $\varphi_c^{(w/z)}$, see
417: Fig.\ \ref{cogSys_fig_gaps}.
418:
419: \item\underline{Reservoir dynamics}\\
420: The reservoir levels of the winning clique
421: depletes slowly, see Eq.~(\ref{cogSys_phidot})
422: and Fig.\ \ref{fig_7sites}, and recovers only
423: once the activity level $x_i$ of a given site has
424: dropped below $x_c$, which defines a site to be
425: active when $x_i>x_c$. The factor $(1-x_i/x_c)$ occurring in
426: the reservoir growth process, see the r.h.s.\ of
427: (\ref{cogSys_phidot}), serves for a stabilization of the
428: transition between two subsequent memory
429: states. When the activity level $x_i$ of a given center
430: $i$ drops below $x_c$, it cannot be reactivated immediately;
431: the reservoir cannot fill up again for $x_i\simeq x_c$,
432: due to the $(1-x_i/x_c)$ in (\ref{cogSys_phidot}).
433:
434: \item\underline{Separation of time scales}\\
435: A separation of time scales is obtained when the
436: $\Gamma_\varphi^\pm$ are much smaller than the
437: typical strength of an active excitatory link,
438: i.e.\ of a typical $w_{ij}>0$,
439: leading to transient state dynamics.
440: Once the reservoir of a winning clique
441: is depleted, it looses, via $f_z(\varphi)$, its ability to
442: suppress other sites and the mutual intra-clique
443: excitation is suppressed via $f_w(\varphi)$.
444:
445: \item\underline{Absence of stationary solutions}\\
446: There are no stationary solutions with
447: $\dot x_i=0=\dot\varphi$ ($i=1,\dots,N$)
448: for Eqs.\ (\ref{cogSys_xdot}) and (\ref{cogSys_phidot}),
449: whenever $\Gamma_\varphi^\pm>0$ do not vanish
450: and for any non-trivial coupling functions
451: $f_{w/z}(\varphi)\in[0,1]$.
452:
453: When decoupling the activities and the reservoir
454: by setting $f_{w/z}(\varphi)\equiv 1$ one
455: obtains stable attractors with
456: $x_i=1/0$ and $\varphi_i=0/1$ for sites
457: belonging/not-belonging to the winning clique,
458: compare Fig.\ \ref{fig_escapeManifold}.
459: %
460: \end{itemize}
461: %
462: In Fig.\ \ref{fig_7sites} the
463: transient state dynamics resulting from
464: Eqs.\ (\ref{cogSys_xdot}-\ref{cogSys_z_t}) is illustrated.
465: Presented in Fig.\ \ref{fig_7sites} are data for the autonomous
466: dynamics in the absence of external sensory signals, we will
467: discuss the effect of external stimuli further below.
468: We present in Fig.\ \ref{fig_7sites} only data for a
469: very small network, containing seven sites, which can
470: be easily represented graphically. We have also performed
471: extensive simulations for very large
472: networks, containing several thousands of sites,
473: and found stable transient state dynamics.
474:
475: %----------------------------------------------------%
476: \subsection{Role of the reservoir}
477: %----------------------------------------------------%
478:
479: The dynamical system discussed here represents in first place
480: a top-down approach to cognitive systems and a
481: one-to-one correspondence with cortical structures
482: is not intended. The setput is however inspired
483: by biological analogies and we may identify
484: the sites $i$ of the artificial neural network
485: described by Eq.\ (\ref{cogSys_xdot})
486: not with single neurons, but with neural
487: assemblies or neural centers. The reservoir variables
488: $\varphi_i(t)$ could therefore be interpreted as
489: effective fatigue processes taking place in continuously
490: active neural assemblies, the winning coalisions.
491:
492: It has been been proposed \cite{malsburg86},
493: that the neural coding used for the binding
494: of heterogeneous sensory information in terms
495: of distinct and recognizable objects might
496: be temporal in nature. Within this temporal
497: coding hypothesis, which has been investigated
498: experimentally \cite{gray89}, neural assemblies
499: fire in phase, {\it viz} synchronous, when
500: defining the same object and asynchronous when
501: encoding different objects. There is a close
502: relation between objects and memories in
503: general. An intriguing possibility is therefore
504: to identify the memories of the transient-state
505: network investigated in the present approach
506: with the synchronous firing neurons of the
507: temporal coding theory. The winning coalition
508: is characterized by high reservoir levels which
509: would then correspond to the degree of
510: synchronization within the temporal encoding
511: paradigm and the reservoir depletion time
512: $\sim1/\Gamma_\varphi^-$
513: would correspond to the decoherence
514: time of the object binding neurons.
515:
516: We note, that this analogy can however
517: not be carried to far, since synchronization
518: is at its basis a cooperative effect, the
519: reservoir levels describing on the other
520: side single-unit properties. In terms
521: of a popular physics phrase one might
522: speak of a `poor man's' approach to
523: synchronization, via coupling to a fatigue
524: variable.
525:
526: %----------------------------------------------------%
527: \subsection{Dissipative dynamics}
528: %----------------------------------------------------%
529:
530: The reason for the observed numerical and dynamical
531: robustness can be traced back to its relaxational nature.
532: For short time scales we can consider the
533: reservoir variables $\left\{\varphi_i\right\}$ to be approximatively
534: constant and the system relaxes into the next
535: clique attractor ruin. Once close to a transient attractor, the
536: $\left\{x_i\right\}$ are essentially constant, {\it viz} close to
537: one/zero and the reservoir slowly depletes. The dynamics
538: is robust against noise, as fluctuations affect only details
539: of both relaxational processes, but not their overall behavior.
540:
541: To be precise we note, that the phase space contracts with respect
542: to the reservoir variables, namely
543: $$
544: \sum_i {\partial \dot\varphi_i\over\partial\varphi_i}
545: \ =\ -\sum_i\left[\Gamma_\varphi^+(1-x_i/x_c)\Theta(x_c-x_i)
546: +\Gamma_\varphi^-\Theta(x_i-x_c)
547: \right] \ \le\ 0,
548: \quad\qquad \forall x_i\in[0,1]~,
549: $$
550: where we have used (\ref{cogSys_phidot}). We note
551: that the diagonal contributions to the link matrices
552: vanish, $z_{ii}=0=w_{ii}$, and therefore
553: $\partial r_i/\partial x_i =0$. The phase space
554: contracts consequently also with respect to the
555: activities,
556: $$
557: \sum_i {\partial \dot x_i\over\partial x_i}
558: \ =\ \sum_i \,\Big[\,\Theta(-r_i) -\Theta(r_i)\, \Big]\, r_i
559: \ \le\ 0~,
560: $$
561: where we have used (\ref{cogSys_xdot}). The system is therefore
562: strictly dissipative, leading to the observed numerically robust
563: behavior.
564:
565: %----------------------------------------------------%
566: \subsection{Strict transient state dynamics}
567: \label{sec_strict}
568: %----------------------------------------------------%
569:
570: The self-generated transient state dynamics shown in
571: Fig.\ \ref{fig_7sites} exhibits well
572: characterized plateaus in the $x_i(t)$, since
573: small values have been used for the
574: depletion and the growth rate of the reservoir,
575: $\Gamma_\varphi^-=0.005$ and $\Gamma_\varphi^+=0.015$.
576: The simulations
577: presented in Fig.\ \ref{fig_7sites} were performed
578: using $w_{ij}=0.12$ for all non-zero excitatory
579: interconnections.
580:
581: We define a dynamical system to have `strict
582: transient state dynamics' if there exists a set of
583: control parameters allowing to turn the transient
584: states adiabatically into stable attractors.
585: Eqs.\ (\ref{cogSys_xdot}-\ref{cogSys_z_t})
586: fulfill this requirements, for
587: $\Gamma_\varphi^-\to 0$ the average duration $\bar t$ of the
588: steady-stated plateaus observed in
589: Fig.\ \ref{fig_7sites} diverges.
590:
591: Alternatively, by selecting appropriate values
592: for $\Gamma_\varphi^-$ and $\Gamma_\varphi^+$, it is possible
593: to regulate the `speed' of the transient state
594: dynamics, an important consideration
595: for applications. For a working cognitive system, such
596: as the brain, it is enough that the transient states
597: are stable just for a certain minimal period needed to
598: identify the state and to act upon it.
599: Anything longer would just be a `waste of time'.
600:
601: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
602: \begin{figure}[t]
603: \centerline{
604: \includegraphics*[width=0.70\textwidth]{escapeManifold.eps}
605: }
606: \caption{The attractors of the original network,
607: {\it viz} when the coupling to the reservoir
608: is turned off by setting $f_{w/z}(\varphi)\equiv 1$,
609: correspond to
610: $x_i=1/0$ and $\varphi_i=0/1$ ($i=1,\dots, N$) for
611: member/non-members of the winning clique. A finite
612: coupling to the local reservoirs $\varphi_i$
613: leads to orbit $\left\{x_i(t),\varphi_i(t)\right\}$
614: which are attracted by the attractor ruins for
615: short time scales and repelled for long time scales.
616: This is due to a separation of time scales,
617: as the time evolution of the reservoirs $\varphi_i(t)$
618: occurs on time scales substantially slower than that of
619: the primary dynamical variables $x_i(t)$.
620: }
621: \label{fig_escapeManifold}
622: \end{figure}
623: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
624:
625: %----------------------------------------------------%
626: \subsection{Universality}
627: %----------------------------------------------------%
628:
629: We note that the mechanism for the generation of stable
630: transient state dynamics proposed here is universal in
631: the sense that it can be applied to a wide range
632: of dynamical systems in a frozen state, i.e.\ which are
633: determined by attractors and cycles.
634:
635: Physically, the mechanism we propose here is to embed the
636: phase space $\left\{ x_i\right\}$
637: of an attractor network into a larger space,
638: $\left\{ x_i,\ \varphi_j\right\}$, by coupling to
639: additional local slow variables $\varphi_i$.
640: Stable attractors are transformed into attractor
641: ruins since the new variables allow the system to
642: escape the basin of the original attractor
643: $\left\{ x_i=1/0,\ \varphi_j=0/1\right\}$
644: (for in-clique/out-of-clique sites)
645: via local escape processes which deplete the respective
646: reservoir levels $\varphi_i(t)$. Note, that the
647: embedding is carried out via the reservoir
648: functions $f_{z/w}(\varphi)$ in Eq.\ (\ref{cogSys_ri})
649: and that the reservoir variables keep a
650: slaved dynamics (\ref{cogSys_phidot}) even
651: when the coupling is turned off by setting
652: $f_{z/w}(\varphi)\to 1$ in Eq.\ (\ref{cogSys_ri}).
653:
654: This mechanism is
655: illustrated in Fig.\ \ref{fig_escapeManifold}.
656: Locality is an important ingredient for this
657: mechanism to work. The trajectories would otherwise
658: not come close to any of the attractor ruins again,
659: {\it viz} to the original attractors,
660: being repelled by all of them with similar
661: strengths and fluctuating dynamics of the
662: kind illustrated in Fig.\ \ref{fig_transStates}
663: would result.
664:
665: %----------------------------------------------------%
666: \subsection{Cycles and time reversal symmetry}
667: %----------------------------------------------------%
668:
669: The systems illustrated
670: in Figs.\ \ref{fig_7sites} and \ref{fig_9sites}
671: are very small and the transient state dynamics
672: soon settles into a cycle of attractor ruins,
673: since there are no incoming sensory signals
674: considered in the respective simulations.
675: For networks containing a larger number of sites, the
676: number of attractors can be however very large and such
677: the resulting cycle length. We performed simulations
678: for a 100-site network, containing 713 clique-encoded
679: memories. We found no cyclic behavior even for sequences
680: of transient states containing up to 4400 transient states.
681: We note, that the system does not necessarily retrace its
682: own trajectory once a given clique is stabilized for a second
683: time, an event which needs to occur in any finite system. The
684: reason being, that the distribution of reservoir levels
685: is in general different when a given clique is revisited
686: for a second time.
687:
688: We note, that time reversal symmetry is `spontaneously'
689: broken in the sense that repetitive transient state dynamics of
690: type
691: \medskip
692:
693: \centerline{
694: (clique A)\ $\to$\ (clique B)\ $\to$\ (clique A)\ $\to$\ (clique B)\ $\to$\ \dots
695: }
696:
697: \medskip\noindent
698: does generally not arise. The reason is simple. Once the
699: first clique is deactivated its respective reservoir
700: levels need a certain time to fill up again,
701: compare Fig.\ \ref{fig_7sites}. Time reversal symmetry would be
702: recovered however in the limit
703: $\Gamma_\varphi^+ \gg \Gamma_\varphi^-$,
704: i.e.\ when the reservoirs would be refilled
705: much faster than depleted.
706:
707: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
708: \begin{figure}[t]
709: \centerline{
710: \includegraphics*[width=0.30\textwidth]{9sites_cyclic.eps}\hspace{1ex}
711: \lower 3ex \hbox{\includegraphics*[width=0.55\textwidth]{A9_cyclic.eps}}
712: }
713: \caption{Geometry and simulation results for
714: a cyclic 9-site network with symmetric excitatory
715: links $w_{ij}=w_{ji}$.\newline
716: Left: The links with $w_{i,j}>0$, containing six cliques,
717: (0,1), (1,2,3) (which is highlighted),
718: (3,4), (4,5,6), (6,7) and (7,8,0). \newline
719: Right:
720: As a function of time, the activities $x_i(t)$
721: for the cyclic transient state dynamics $(1,2,3)\rightarrow(7,8,0)
722: \rightarrow(4,5,6,)\rightarrow\dots$, for the
723: parameters values see Sect.\ \ref{sec_strict}. Both
724: directions (clockwise/anticlockwise) of `rotation' are
725: dynamically possible and stable,
726: the actual direction being determined by the dynamical
727: initial conditions.
728: \label{fig_9sites}
729: }
730: \end{figure}
731: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
732: %----------------------------------------------------%
733: \subsection{Reproducible sequence generation}
734: %----------------------------------------------------%
735:
736: Animals need to generate sequences of neural activities
737: for a wide range of purposes, e.g.\ for movements or for
738: periodic internal muscle contractions, the heart-beat
739: being a prime case. These sequences need to
740: be successions of well defined firing patterns,
741: usable to control actuators, {\it viz} the muscles.
742: The question then arrises under which condition a
743: dynamical system generates reproducible sequences
744: of well defined activity patterns, i.e.\
745: controlled time series of transient states \cite{huerta01,rabinovich06b}.
746:
747: There are two points worth noting in this context.
748: %
749: \begin{description}
750: %
751: \item[i.] The dynamics described by
752: Eqs.\ (\ref{cogSys_xdot}-\ref{cogSys_phidot})
753: works fine for randomly selected link matrices
754: $w_{ij}$ which may, or may not change
755: with time passing. In particular one can
756: select the cliques specifically in order to induce
757: the generation of a specific succession of
758: transient states, an example is presented
759: in Fig.\ \ref{fig_9sites}. The network is
760: capable, as a matter of principle
761: to generate robustly large numbers of different
762: sequences of transient states. For geometric
763: arrangements of the networks sites, and of the
764: links $w_{ij}$, one finds waves of transient
765: states sweeping through the system.
766: %
767: \item[ii.] In Sect.\ \ref{sec_Autonomous_online_learning}
768: we will discuss how appropriate $w_{ij}$ can
769: be learned from training patterns presented to
770: the network by an external teacher. We will
771: concentrate in Sect.\ \ref{sec_Autonomous_online_learning}
772: on the training and learning
773: of individual memories, {\it viz} of cliques,
774: but suitable sequences of training patters could be
775: used also for learning temporal sequences of
776: memories.
777: %
778: \end{description}
779:
780: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
781: \section{Autonomous online learning}
782: \label{sec_Autonomous_online_learning}
783: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
784:
785: An external stimulus, $\{b_i^{(ext)}(t)\}$,
786: influences the activities $x_i(t)$ of the respective neural
787: centers. This corresponds to a change of the respective
788: growth rates $r_i$,
789: %
790: \begin{equation}
791: r_i \ \to \ r_i \,+\,f_z(\varphi_i)\,b_i^{(ext)}(t)~,
792: \label{cogSys_stim}
793: \end{equation}
794: %
795: compare Eq.\ (\ref{cogSys_ri}), where $f_z(\varphi_i)$
796: is an appropriate coupling function, depending on the
797: local reservoir level $\varphi_i$. When the
798: effect of the external stimulus is strong,
799: namely when $f_z b_i^{(ext)}$ is strong,
800: it will in general lead to an activation
801: $x_i\to1$ of the respective neural center $i$.
802: A continously active stimulus does not convey
803: new information and should, on the other
804: hand, lead to habituation, having a reduced
805: influence on the system. A strong, continously
806: present stimulus leads to a prolonged high
807: activity level $x_i\to1$ of the involved
808: neural centers, leading via (\ref{cogSys_phidot})
809: to a depletion of the respective reservoir
810: levels, on a time scale given by the
811: inverse reservoir depletion rate,
812: $1/\Gamma_\varphi^-$.
813: Habituation is then mediated
814: by the coupling function $f_z(\varphi_i)$
815: in (\ref{cogSys_stim}), since
816: $f_z(\varphi_i)$ becomes very small for
817: $\varphi_i\to0$, compare
818: Fig.\ \ref{cogSys_fig_gaps}.
819: The effect of habituation incorporated in
820: (\ref{cogSys_stim}) therefore allows the system to
821: turn its `attention' to other competing stimuli,
822: with novel stimuli having a higher chance to affect the
823: ongoing transient state dynamics.
824:
825: We now provide a set of learning rules allowing the system
826: to acquire new patterns on the fly, {\it viz} during
827: its normal phase of dynamical activity. The alternative,
828: modeling networks having distinct periods
829: of learning and of performance, is of widespread
830: use for technical applications of neural networks, but
831: is not of interest in our context of continuously active
832: cognitive systems.
833:
834: %----------------------------------------------------%
835: \subsection{Short- and long term synaptic plasticities}
836: %----------------------------------------------------%
837:
838: There are two fundamental considerations for the choice
839: of synaptic plasticities adequate for neural networks with
840: transient state dynamics.
841: \begin{itemize}
842: \item Learning is a very slow process without a short term memory.
843: Training patterns need to be presented to the network over and
844: over again until substantial synaptic changes are induced \cite{arbib02}.
845: A short term memory can speed-up the learning process substantially,
846: as it stabilizes external patterns and hence
847: gives the system time to consolidate long term
848: synaptic plasticity.
849:
850: \item Systems using sparse coding are based on a strong
851: inhibitory background, the average inhibitory link-strength
852: $|z|$ is substantially larger than the average
853: excitatory link strength $\bar w$,
854: $$
855: |z|\ \gg\ \bar w~.
856: $$
857: It is then clear that gradual learning is effective only when
858: it affects dominantly the excitatory links: Small changes
859: of large parameters do not lead to new transient
860: attractors, nor do they influence the cognitive dynamics
861: substantially.
862: \end{itemize}
863: %
864: It then follows, that it is convenient to split the synaptic plasticities
865: into two parts,
866: %
867: \begin{equation}
868: w_{ij}\ =\ w_{ij}(t)\ \equiv\ w_{ij}^{S}(t)\,+\, w_{ij}^{L}(t)~,
869: \label{cogSys_w_S_L}
870: \end{equation}
871: %
872: where the $w_{ij}^{S/L}$ correspond to the short term and to the
873: long term synaptic plasticities respectively.
874:
875: %----------------------------------------------------%
876: \subsection{Negative baseline}
877: \label{sec_neg_baseline}
878: %----------------------------------------------------%
879:
880: Eq.\ (\ref{cogSys_z_t}), $z_{ij} = -|z|\,\Theta(-w_{ij})$,
881: implies that the inhibitory link strength is either
882: zero or $-|z|$, but is not changed directly during learning,
883: in accordance to above discussion.
884: We may therefore consider two kinds of
885: `excitatory links strengths':
886: %
887: \begin{itemize}
888: \item \underline{\sl Active}: An active $w_{i,j}>0$
889: is positive and enforces $z_{ij}=0$
890: for the same link, via Eq.\ (\ref{cogSys_z_t}).
891: \item \underline{\sl Inactive}: An inactive $w_{i,j}<0$
892: is slightly negative, we use
893: $w_{i,j}=W_L^{(min)}<0$ as a default.
894: It enforces $z_{ij}=-|z|$
895: for the same link, via Eq.\ (\ref{cogSys_z_t})
896: and does not contribute to the dynamics,
897: since the excitatory links enter as
898: $\theta(w_{i,j})$ in (\ref{cogSys_ri}).
899: \end{itemize}
900: %
901: When $w_{i,j}$ acquires, during learning, a positive value,
902: the corresponding inhibitory link
903: $z_{ij}$ is turned off via Eq.\ (\ref{cogSys_z_t})
904: and the excitatory link $w_{i,j}$ determines
905: the value of the respective term,
906: $f_w(\varphi_i) \Theta(w_{ij}) w_{i,j}
907: + z_{i,j}f_z(\varphi_j)$,
908: in Eq.\ (\ref{cogSys_ri}).
909: We have used a small negative baseline of
910: $W_L^{(min)}=-0.01$ throughout the simulations.
911:
912: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
913: \begin{figure}[t]
914: \centerline{
915: \includegraphics*[width=0.70\textwidth]{LL7_learn_S.eps}
916: }
917: \caption{The time evolution of the short
918: term-memory, for some selected
919: links $w_{i,j}^S$ and the network illustrated
920: in Fig.\ \ref{fig_7sites}, without the link (3,6).
921: The transient states are
922: $(0,1)\rightarrow(4,5,6)
923: \rightarrow(1,2,3)
924: \rightarrow(3,6)
925: \rightarrow(0,6)
926: \rightarrow(0,1)
927: $.
928: An external stimulus at
929: sites (3) and (6) acts for $t\in[400,410]$ with
930: strength $b^{(ext)}= 3.6$.
931: \label{cogSys_fig_7_AI_learn_S}
932: }
933: \end{figure}
934: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
935:
936:
937: %-------------------------------------%
938: \subsection{Short term memory dynamics}
939: %-------------------------------------%
940:
941: It is reasonable to have a maximal possible value
942: $W_S^{(max)}$ for the short term synaptic
943: plasticities. We consider therefore
944: the following Hebbian-type learning rule:
945: %
946: \begin{eqnarray}
947: \label{cogSys_w_S_dot}
948: \dot w_{ij}^S(t) & =&
949: \Gamma_{S}^+
950: \left(W_S^{(max)}-w_{ij}^S\right)
951: f_z(\varphi_i) f_z(\varphi_j) \,
952: \Theta(x_i-x_c) \Theta(x_j-x_c) \\
953: &-& \Gamma_{S}^-\,w_{ij}^S~.
954: \nonumber
955: \end{eqnarray}
956: %
957: $w_{ij}^S(t)$ increases rapidly, with rate $\Gamma_S^+$,
958: when both the pre- and the post-synaptic neural
959: centers are active, {\it viz} when their
960: respective activities are above $x_c$.
961: Otherwise it decays to zero, with a rate $\Gamma_S^-$.
962: The coupling functions $f_z(\varphi)$ preempt
963: prolonged self-activation of the short term
964: memory. When the pre- and the post-synaptic centers
965: are active long enough to deplete their respective
966: reservoir levels, the short term memory is shut-off
967: via the $f_z(\varphi)$.
968: We have used $\Gamma_S^+=0.1$, $\Gamma_S^-=0.0005$ and
969: $W_S^{(max)}=0.02$ and $x_c=0.85$ throughout the simulations.
970:
971: In Fig.\ \ref{cogSys_fig_7_AI_learn_S} we present the time evolution
972: of some selected $w_{ij}^S(t)$, for a simulation using the
973: network illustrated in Fig.\ \ref{fig_7sites}. The short term
974: memory is activated in three cases:
975: %
976: \begin{itemize}
977: \item When an existing clique, {\it viz} a clique encoded in
978: the long term memory $w_{ij}^L$, is activated,
979: as it is the case of (0,1) for the data presented in
980: Fig.\ \ref{cogSys_fig_7_AI_learn_S}, the respective
981: intra-clique $w_{ij}^S$ are also activated. This
982: behavior is a side effect since, for the parameter values
983: chosen here, the magnitude of the short term link strengths
984: is substantially smaller than those of the long term
985: link strengths.
986: %
987: \item During the transient state dynamics there is a certain
988: overlapping of a currently active clique with the subsequent
989: active clique. For this short time span the short term plasticities
990: $w_{ij}^S$ for synapses linking these two cliques get
991: activated. An example is the link (2,4) for the simulation
992: presented in Fig.\ \ref{cogSys_fig_7_AI_learn_S}.
993: %
994: \item When external stimuli act on two sites not connected by
995: an excitatory long term memory link $w_{ij}^L$, the
996: short term plasticity $w_{ij}^S$ makes a qualitative difference.
997: It transiently stabilizes the corresponding link and the
998: respective link becomes a new clique $(i,j)$ either by itself,
999: or as part of an enlarged and already existing clique. An example
1000: is the link (3,6) for the simulation
1001: presented in Fig.\ \ref{cogSys_fig_7_AI_learn_S}. Note however that,
1002: without subsequent transferal into the long term memory, these
1003: new states would disappear with a rate $\Gamma_S^-$ once the
1004: causing external stimulus is gone.
1005: \end{itemize}
1006: %
1007: The last point is the one of central importance,
1008: as it allows for the temporal stabilization of new
1009: patterns present in the sensory input stream.
1010:
1011: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
1012: \begin{figure}[t]
1013: \centerline{
1014: \includegraphics*[width=0.70\textwidth]{LL7_learn_L.eps}
1015: }
1016: \caption{The time evolution of the long term memory,
1017: for some selected links $w_{i,j}^L$ and the network illustrated
1018: in Fig.\ \ref{fig_7sites}, without the link (3,6).
1019: The transient states are
1020: $(0,1)\rightarrow(4,5,6)
1021: \rightarrow(1,2,3)
1022: \rightarrow(3,6)
1023: \rightarrow(0,6)
1024: \rightarrow(0,1)
1025: $.
1026: An external stimulus at
1027: sites (3) and (6) acts for $t\in[400,410]$ with
1028: strength $b^{(ext)}= 3.6$. The stimulus pattern
1029: (3,6) has been learned by the system, as the
1030: $w_{3,6}$ and $w_{6,3}$ turned positive during the
1031: learning-interval $\approx [400,460]$. The learning
1032: interval is substantially longer than the bare
1033: stimulus length due to the activation of the short term
1034: memory. The decay of certain $w_{ij}^L$ in the
1035: absence of an external stimulus is due to forgetting
1036: (\ref{cogSys_w_L_dot_opt}), which should normally
1037: be a very weak effect, but which has been choose here
1038: to be a sizeable $\Gamma_L^-=0.1$,
1039: for illustrational purposes
1040: }
1041: \label{cogSys_fig_7_AI_learn_L}
1042: \end{figure}
1043: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
1044:
1045: %------------------------------------%
1046: \subsection{Long term memory dynamics}
1047: %------------------------------------%
1048:
1049: Information processing dynamical systems retain
1050: their functionalities only when they keep their
1051: dynamical properties within certain regimes,
1052: they need to regulate their own working point.
1053: For the type of systems discussed here, exhibiting
1054: transient-state dynamics, the working point
1055: is, as discussed in the introduction,
1056: defined as the time $\Delta t$ the system
1057: needs for a transition from one quasi-stationary
1058: state to the subsequent, relative to the length
1059: $\bar t$ of the individual quasi-stationary states,
1060: which is given by $1/\Gamma_\varphi^-$.
1061:
1062: The cognitive information
1063: processing within neural networks occurs on
1064: short to intermediate time scales. For these
1065: processes to work well the mean overall synaptic
1066: plasticities, {\it viz} the average strength of the
1067: long term memory links $w_{ij}^{L}$, needs to be
1068: regulated homeostatically.
1069: The average magnitude of the
1070: growth rates $r_i$, see Eq.\ (\ref{cogSys_ri}),
1071: determines the time $\Delta t$ needed to complete a transition
1072: from one winning clique to the next transient state.
1073: It therefore constitutes a central quantity regulating
1074: the working point of the system, since
1075: $\bar t\sim 1/\Gamma_\varphi^-$ is fixed, the
1076: reservoir depletion rate $\Gamma_\varphi^-$
1077: is not affected by learning processes which
1078: affect exclusively the inter-neural synaptic strengths.
1079:
1080: The bare growth rates $r_i(t)$ are quite strongly
1081: time dependent, due to the time-dependence of the postsynaptic
1082: reservoirs entering the reservoir function
1083: $f_w(\varphi_i)$, see Eq.\ (\ref{cogSys_ri}).
1084: The effective incoming synaptic signal strength
1085: %
1086: \begin{equation}
1087: %
1088: \tilde r_i \,=\, \sum_{j}\Big[ w_{i,j}
1089: \,+\, z_{i,j}f_z(\varphi_j)\Big]x_j~,
1090: \label{cogSys_tilde_r}
1091: \end{equation}
1092: %
1093: which is independent of the post-synaptic
1094: reservoir $\varphi_i$, is a more convenient local
1095: control parameter. The working point of the cognitive
1096: system is optimal when the effective
1097: incoming signal is, on the average, of comparable
1098: magnitude $r^{(opt)}$ for all sites,
1099: %
1100: \begin{equation}
1101: \tilde r_i\ \to\ r^{(opt)}~.
1102: \label{cogSys_r_to_r_opt}
1103: \end{equation}
1104: %
1105: The long term memory has two tasks: To extract and
1106: encode patterns present in the external stimulus,
1107: Eq.\ (\ref{cogSys_stim}), via unsupervised learning
1108: and to keep the working point of the dynamical system
1109: in its desired range. Both tasks can be achieved by a
1110: single local learning rule,
1111: %
1112: \begin{eqnarray}
1113: \label{cogSys_w_L_dot_opt}
1114: \dot w_{ij}^L(t) & =&
1115: \Gamma_{L}^{(opt)}
1116: \Delta \tilde r_i \Big[\,
1117: \left(w_{ij}^L-W_L^{(min)}\right) \Theta(-\Delta \tilde r_i) +
1118: \Theta(\Delta \tilde r_i)
1119: \,\Big] \\ &&\cdot
1120: \,\Theta(x_i-x_c)\,\Theta(x_j-x_c),
1121: %\qquad\qquad\qquad\qquad
1122: %\Delta \tilde r_i\,=\,r^{(opt)}-\tilde r_i~.
1123: \nonumber \\
1124: & - & \Gamma_L^- \, d(w_{ij}^L)\, \Theta(x_i-x_c)\,\Theta(x_c-x_j)~,
1125: \label{cogSys_w_L_dot_decay}
1126: \end{eqnarray}
1127: %
1128: %
1129: where $\Delta \tilde r_i\,=\,r^{(opt)}-\tilde r_i$.
1130: For the numerical simulations we used
1131: $\Gamma_L^{(opt)}=0.0008$,
1132: $W_L^{(min)}=-0.01$ and $r^{(opt)}=0.2$.
1133: We now comment on some properties of these
1134: evolution equations for $w_{ij}^L(t)$:
1135: %
1136: \begin{itemize}
1137: %
1138: \item\underline{Hebbian learning}\newline
1139: The learning rule (\ref{cogSys_w_L_dot_opt})
1140: is local and of Hebbian type.
1141: Learning occurs only when the pre- and the post-synaptic
1142: neuron are active, {\it viz} when their respective
1143: activity levels are above the threshold $x_c$.
1144: Weak forgetting, i.e.\ the decay
1145: of seldom used links is governed by
1146: (\ref{cogSys_w_L_dot_decay}). The function
1147: $d(w_{ij}^L)$ determines the functional dependence
1148: of forgetting on the actual synaptic strength,
1149: we have used $d(w_{ij}^L)=\theta(w_{ij}^L)w_{ij}^L$
1150: for simplicity.
1151: %
1152: \item\underline{Synaptic competition}\newline
1153: When the effective incoming signal $\tilde r_i$
1154: is weak/strong, relative to
1155: the optimal value $r^{(opt)}$, the active links
1156: are reinforced/weakened, with $W_L^{(min)}$ being the
1157: minimal value for the $w_{ij}$. The baseline $W_L^{(min)}$
1158: is slightly negative, compare
1159: Figs.\ \ref{cogSys_fig_gaps} and \ref{cogSys_fig_7_AI_learn_L}.
1160:
1161: The Hebbian-type learning then takes place in the form of
1162: a temporal competition among incoming synapses - frequently
1163: active incoming links will gain strength, on the average,
1164: on the expense of rarely used links.
1165: %
1166: \item\underline{Fast learning of new patterns}\newline
1167: In Fig.\ \ref{cogSys_fig_7_AI_learn_L} the time evolution
1168: of some selected $w_{ij}^L$ is presented.
1169: A simple input pattern is learned by the
1170: network. In this simulation the learning parameter
1171: $\Gamma_{L}^{(opt)}$ has been set to a quite large value
1172: such that the learning occurs in one step (fast learning).
1173: %
1174: \item\underline{Suppression of runaway synaptic growth}\newline
1175: When a neural network is exposed repeatedly to
1176: the same, or to similar external stimuli, unsupervised
1177: learning generally then leads to uncontrolled growth
1178: of the involved synaptic strengths. This phenomena,
1179: termed `runaway synaptic growth' can also occur
1180: in networks with continuous self-generated activities,
1181: when similar activity patterns are auto-generated
1182: over and over again. Both kinds of synaptic runaway-growth
1183: is suppressed by the proposed link-dynamics (\ref{cogSys_w_L_dot_opt}).
1184: \item\underline{Negative baseline}\newline
1185: Note that $w_{ij}=w_{ij}^S+w_{ij}^L$
1186: enters the evolution equation (\ref{cogSys_ri})
1187: as $\theta(w_{ij})$. We can therefore distinguish
1188: between active ($w_{ij}>0$) and inactive
1189: ($w_{ij}<0$) configuration, compare
1190: Fig.\ \ref{cogSys_fig_gaps}. The negative
1191: baseline $W_L^{(min)}<0$ entering
1192: (\ref{cogSys_w_L_dot_opt}) then allows
1193: for the removal of positive links and
1194: provides a barrier against small random
1195: fluctuations, compare Sect.\ \ref{sec_neg_baseline}.
1196: %
1197: \end{itemize}
1198: %
1199: During a transient state we have $x_i\to1$ for all vertices
1200: belonging to the winning coalition and $x_j\to0$ for
1201: all out-of-clique sites, leading to
1202: $$
1203: \tilde r_i\ \approx\ \sum_{j}\ w_{i,j}, \qquad\quad j\in\mbox{active\ sites}~,
1204: $$
1205: compare Eq.\ (\ref{cogSys_tilde_r}). The working-point optimization
1206: rule (\ref{cogSys_r_to_r_opt}),
1207: $ \tilde r_i \to r^{(opt)}$ is therefore equivalent to a local
1208: normalization condition enforcing the sum of active incoming link-strengths
1209: to be constant, i.e.\ site independent. This rule is closely related
1210: to a mechanism of self regulation of the average firing rate of cortical
1211: neurons proposed by Bienenstock, Cooper and Munro \cite{bienenstock82}.
1212:
1213: %--------------------------%
1214: \subsection{Online learning}
1215: \label{subsec_online_learning}
1216: %--------------------------%
1217:
1218: The neural network we consider here is continously active,
1219: independently of whether there is sensory input via
1220: Eq.\ (\ref{cogSys_stim}) or not. The reason being,
1221: that the evolution equations (\ref{cogSys_xdot})
1222: and (\ref{cogSys_phidot}) generate a never ending
1223: time series of transient states. It is a central
1224: assumption of the present study, that continuous and
1225: self-generated neural activity is a condition
1226: sine qua no for modeling overall brain activity
1227: or for developing autonomous cognitive systems
1228: \cite{gros07}.
1229:
1230: The evolution equations for the synaptic
1231: plasticities, namely (\ref{cogSys_w_S_dot})
1232: for the short-term memory and
1233: (\ref{cogSys_w_L_dot_opt}) for the
1234: long-term memory are part of the dynamical
1235: system, {\it viz} they determine the time
1236: evolution of $w_{ij}^S(t)$ and of
1237: $w_{ij}^L(t)$ at all times, irrespectively of
1238: whether external stimuli are presented
1239: to the network via (\ref{cogSys_stim})
1240: or not. The evolution equations for the
1241: synaptic plasticities need therefore
1242: to fulfill, quite in general for a
1243: continously active neural network,
1244: two conditions:
1245: %
1246: \begin{description}
1247:
1248: \item[(a)] Under training conditions, namely when
1249: input patterns are presented to the
1250: system via (\ref{cogSys_stim}), the
1251: system should be able to modify the
1252: synaptic link strength accordingly,
1253: such that the training patterns
1254: are stored in the form of new memories,
1255: {\it viz} cliques representing
1256: attractor ruins and leading to quasistationary
1257: states.
1258: \item[(b)] In the absence of input the
1259: ongoing transient-state dynamics
1260: will lead constantly to synaptic
1261: modifications, via (\ref{cogSys_w_S_dot})
1262: and (\ref{cogSys_w_L_dot_opt}). Theses
1263: modification may not induce
1264: qualitative changes, such as the
1265: the autonomous destruction of existing
1266: memories or the spontaneous generation
1267: of spurious new memories. New memories
1268: should be acquired exclusively via
1269: training by external stimuli.
1270: \end{description}
1271: %
1272: In the following section we will present simulations
1273: in order to investigate these points. We find
1274: that the evolution equations formulated in this
1275: study conform with both conditions (a) and (b) above,
1276: due to the optimization principle for the
1277: long-term synaptic plasticities in
1278: Eq.\ (\ref{cogSys_w_L_dot_opt}).
1279:
1280:
1281: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
1282: \begin{table}[b]
1283: \caption{Learning results
1284: for systems with $N$ sites and
1285: $N_{links}$ excitatory links and
1286: $N_2,..,N_6$ cliques containing $2,..,6$ sites.
1287: $N_{tot}$ is the total number memories to
1288: be learned. $N_l$ and $N_{par}$ denote
1289: the number of memories learned completely/partially.
1290: }
1291: \begin{center}
1292: \begin{tabular}{cc|cccccc|cc}
1293: \hline
1294: $ \ N \ $ & $ \ N_{links} \ $ &
1295: $ \ N_2 \ $ & $ \ N_3 \ $ &
1296: $ \ N_4 \ $ & $ \ N_5 \ $ &
1297: $ \ N_6 \ $ & $ \ N_{tot} \ $ &
1298: $ \ N_{l} \ $ & $ \ N_{par} \ $ \\
1299: \hline
1300: 20 & 104 & 1 & 10 & 42 & 11 & 1 & 65 & 60 & 3 \\
1301: 100 & 901 & 26 & 563 & 122 & 2 & 0 & 713 & 704 & 7 \\
1302: \hline
1303: \end{tabular}
1304: \end{center}
1305: \label{tab_results_learning}
1306: \end{table}
1307: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
1308:
1309: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
1310: \section{Simulations}
1311: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
1312:
1313: We have performed extensive simulations
1314: of the dynamics of the network with ongoing learning, for
1315: systems with up to several thousands of sites. We found that
1316: the dynamics remains long-term stable even in the presence
1317: of continuous online learning governed by Eqs.\ (\ref{cogSys_w_S_dot})
1318: and (\ref{cogSys_w_L_dot_opt}), exhibiting semi-regular sequences
1319: of winning coalition, as shown
1320: in Figs.\ \ref{fig_7sites}.
1321: The working point is regulated adaptively and no prolonged
1322: periods of stasis or trapped states were observed in
1323: the simulations, neither did periods of rapid or uncontrolled
1324: oscillations occur.
1325:
1326: Any system with a finite number of sites $N$ and a finite
1327: number of cliques settles in the end, in the absence of
1328: external signals, into a cyclic series of transient states.
1329: Preliminary investigations of systems with $N\approx 20-100$
1330: resulted in cycles spanning on the average a finite fraction of
1331: the set of all cliques encoded by the network.
1332: This is a notable result, since the overall number
1333: of cliques stored in the network can easily be
1334: orders of magnitudes larger than the number of sites $N$ itself,
1335: compare Eq.\ (\ref{cogSys_N_z}).
1336: Detailed studies of the cyclic behavior for
1337: autonomous networks will be presented elsewhere.
1338:
1339: %----------------%
1340: \subsection{Learning of new memories}
1341: %----------------%
1342:
1343: Training patterns $\{p_1,..,p_Z\}$ presented to the system externally via
1344: $r_i \to r_i +f_w(\varphi_i)b_i^{(ext)}(t)$, for
1345: $i\in\{p_1,..,p_Z\}$, are learned by the network
1346: via the activation of the short term memory
1347: for the corresponding intra-pattern links.
1348: In Figs.\ \ref{cogSys_fig_7_AI_learn_S} and
1349: \ref{cogSys_fig_7_AI_learn_L} we present a case study.
1350: The ongoing internal transient state dynamics is
1351: interrupted at time $t=400$ by an external signal which
1352: activates the short term memory, see Fig.\ \ref{cogSys_fig_7_AI_learn_S}.
1353: Note that the short term memory is activated both
1354: by external stimuli and internally whenever a given
1355: link becomes active, i.e.\ when both pre- and post-synaptic
1356: sites are active coinstantaneous.
1357: The internal activation does however not
1358: lead to the internal generation of spurious memories, since
1359: internally activated links belong anyhow to one or more
1360: already existing cliques.
1361:
1362: In Fig.\ \ref{cogSys_fig_7_AI_learn_L} we present the
1363: time development of the respective
1364: long term synaptic modifications, $w_{ij}^L(t)$. The
1365: parameters for learning chosen here allow for fast learning,
1366: the pattern corresponding to the external signal,
1367: retained temporarily in the short term
1368: memory, is memorized in one step, {\it viz} the corresponding
1369: $w_{36}^L(t)$ becomes positive before the transition to the
1370: next clique takes place. For practical applications smaller
1371: learning rates might be more suitable, as they allow
1372: to avoid learning of spurious signals generated by
1373: environmental noise.
1374:
1375: In Table \ref{tab_results_learning} we present the results
1376: for the learning of two networks with $N=20$ and $N=100$
1377: from scratch. The initial networks contained only two
1378: connected cliques, in order to allow for a non-trivial initial
1379: transient state dynamics, all other links where inhibitory.
1380: Learning by training and storage of
1381: the externally presented patterns,
1382: using the same parameters as for Figs.\ \ref{cogSys_fig_7_AI_learn_S}
1383: and \ref{cogSys_fig_7_AI_learn_L}, is nearly perfect. The learning
1384: rate can be chosen over a very wide range, as we tested. Here the
1385: training phase was completed, for the 100-site network,
1386: by $t=5\cdot 10^4$. Coming back to the discussion in
1387: section \ref{subsec_online_learning}, we then conclude that
1388: the network fulfills the there formulated condition
1389: (a), being able to store efficiently training patterns
1390: as attractor ruins in the form of cliques.
1391:
1392:
1393: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
1394: \begin{figure}[t]
1395: \centerline{
1396: \includegraphics*[width=0.45\textwidth]{100WW_learn.eps}
1397: \hspace{3ex}
1398: \includegraphics*[width=0.45\textwidth]{100WW_start.eps}
1399: }
1400: \caption{The link-asymmetry $w_{ij}^L-w_{ji}^L$
1401: for the positive $w_{ij}^L$
1402: for a 100-site network with 713 cliques
1403: at time $t=5\cdot 10^5$, corresponding to circa
1404: 4500 transient states.
1405: \newline
1406: Left: After learning from scratch.
1407: Training was finished at $t\approx5\cdot10^4$. \newline
1408: Right: Starting with $w_{i,j}\to0.12$ for
1409: all links belonging to one or more cliques.
1410: \label{cogSys_fig_100_links}
1411: }
1412: \end{figure}
1413: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
1414:
1415: %-------------------------%
1416: \subsection{Link asymmetry}
1417: %-------------------------%
1418:
1419: We note that the Hebbian learning via the
1420: working-point optimization, Eq.\ (\ref{cogSys_w_L_dot_opt}),
1421: leads to the spontaneous generation of asymmetries in the
1422: link matrices, {\it viz} to $w_{ij}^L\ne w_{ji}^L$,
1423: since the synaptic plasticity depends on the
1424: postsynaptic growth rates.
1425:
1426: In Fig.\ \ref{cogSys_fig_100_links} we present,
1427: for two simulations, the distribution
1428: of the link-asymmetry $w_{ij}^L-w_{ji}^L$
1429: for all positive $w_{ij}^L$,
1430: for the 100-site network of Table
1431: \ref{tab_results_learning}, at time $t=5\cdot10^5$.
1432: The distributions shown in Fig.\ \ref{cogSys_fig_100_links}
1433: are particular realizations of steady-state distributions,
1434: {\it viz} they did not change appreciably for
1435: wide ranges of total simulation times.
1436: %
1437: \begin{description}
1438:
1439: \item[(i)] In the first simulation the network had been
1440: learned from scratch. The set of 713 training patterns
1441: were presented to the network for $t\in[0,5\cdot10^4]$.
1442: After that, for $t\in[5\cdot10^4,5\cdot10^5]$
1443: the system evolved freely. A total of 3958 transient
1444: states had been generated at time $t=5\cdot10^5$,
1445: but the system had nevertheless not yet settled into
1446: a cycle of transient states, due
1447: to the ongoing synaptic optimization,
1448: Eqs.\ (\ref{cogSys_w_S_dot}) and (\ref{cogSys_w_L_dot_opt}).
1449: 661 cliques remained at $t=5\cdot10^5$, as the link
1450: competition had led to the suppression of some seldom
1451: used links.
1452:
1453: \item[(ii)]
1454: In the second simulation, uniform and symmetric
1455: starting excitatory links $w_{ij}^L\to 0.12$ had been set
1456: by hand at $t=0$, for all intra-clique links. The same $N=100$
1457: network as in (i) was used and the simulation
1458: ran in the absence of external stimuli. All 713 cliques
1459: were still present at $t=5\cdot10^5$, despite the
1460: substantial reorganization of the link-strength
1461: distribution, from the initial uniform to the
1462: stationary distribution shown in Fig.\ \ref{cogSys_fig_100_links}.
1463: A total of 4123 transient states had been generated
1464: in the course of the simulation, without the
1465: system entering into a cycle.
1466:
1467: \end{description}
1468: %
1469: For both simulations all evolution equations,
1470: namely (\ref{cogSys_xdot}) and (\ref{cogSys_phidot})
1471: for the activities and reservoir levels, as well as
1472: (\ref{cogSys_w_S_dot}) for the short-term memory and
1473: (\ref{cogSys_w_L_dot_opt}) and (\ref{cogSys_w_L_dot_decay})
1474: for the long term memory determined the dynamics
1475: for all times $t\in[0,5\cdot 10^5]$. The difference
1476: between (i) and (ii) being the way the memories are
1477: determined, via training by external stimuli,
1478: Eq.\ (\ref{cogSys_stim}), as in (i)
1479: or by hand as in (ii).
1480:
1481: Comparing the two link distributions shown in
1482: Fig.\ \ref{cogSys_fig_100_links}, we note the overall similarity,
1483: a consequence of the continuously acting
1484: working-point optimization. The main differences
1485: turn-up for small link strengths,
1486: since these two simulations started from opposite
1487: extremes (vanishing/strong initial excitatory links).
1488: The details of the link distribution shown
1489: in Fig.\ \ref{cogSys_fig_100_links} depend
1490: sensitively on the parameters. For the
1491: results show in Fig.\ \ref{cogSys_fig_100_links}
1492: we used for illustrational purposes
1493: $\Gamma_L^-=0.1$, which is a very big
1494: value for a parameter regulating weak forgetting.
1495: We also performed simulations with $\Gamma_L^-=0$,
1496: the other extreme, and found that the
1497: link-asymmetry distribution
1498: was somewhat more scattered.
1499:
1500: Coming back to the discussion in
1501: section \ref{subsec_online_learning}, we then conclude that
1502: the network fulfills the there formulated condition
1503: (b), since essentially no memories acquired during the
1504: training state were destroyed, or spurious new
1505: memories spontaneously creasted,
1506: during the subsequent free evolution.
1507:
1508: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
1509: \section{Conclusions}
1510: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
1511:
1512: We have investigated a series of issues regarding
1513: neural networks with autonomously generated
1514: transient state dynamics. We have presented a
1515: general method allowing to transform an initial
1516: attractor network into a network capable of
1517: generating an infinite time series of transient
1518: states. The resulting dynamical system has strictly
1519: contracting phase space, with a one-to-one adiabatic
1520: correspondence between the transient states and
1521: the attractors of the original network.
1522:
1523: We then have discussed the problem of homeostasis,
1524: namely the need for the system to regulate its
1525: own working point adaptively. We formulated a
1526: simple learning rule for unsupervised local Hebbian-type
1527: learning, which solves the homeostasis problem. We note
1528: here, that this rule, Eq.\ (\ref{cogSys_w_L_dot_opt})
1529: is similar to learning rules shown to
1530: optimize the overall storage capacity
1531: for discrete-time neural networks \cite{chechik01}.
1532:
1533: We have studied a continuous time neural network
1534: model using clique encoding and showed that
1535: this model is very suitable for studying
1536: transient state dynamics in conjunction
1537: with ongoing learning-on-the-fly
1538: for a wide range of learning conditions.
1539: Both fast and slow online learning of new
1540: memories is compatible with the
1541: transient state dynamics self-generated by
1542: the network.
1543:
1544: Finally we turn to the interpretation
1545: of the transient state dynamics.
1546: Examination of a typical time series of
1547: subsequently activated cliques, as the one
1548: shown in Fig.\ \ref{fig_7sites}, reveals that
1549: the sequence of cliques is not random.
1550: Every single clique is connected to its
1551: predecessor via excitatory links, they are
1552: said to be `associatively' connected \cite{gros05}.
1553: The sequence of subsequently active cliques
1554: can therefore be viewed, cum grano salis, as
1555: an `associative thought process' \cite{gros05}. The possible
1556: use of such processes for cognitive information
1557: processing needs, however, yet to be investigated.
1558:
1559: % {\bf Acknowledgments.-}
1560: % I thank R.~Valent\'i\ for carefully reading the manuscript.
1561:
1562: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
1563: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
1564: \section*{References}
1565: \begin{thebibliography}{10}
1566:
1567:
1568: \bibitem{katok95} A. Katok, B. Hasselblatt, L. Mendoza,
1569: {\it ``Introduction to the Modern Theory of Dynamical Systems''},
1570: Cambridge University Press (1995).
1571:
1572: \bibitem{hopfield82} J.J. Hopfield,
1573: {\it ``Neural Networks and Physical Systems with Emergent
1574: Collective Computational Abilities''},
1575: PNAS {\bf 79}, 2554 (1982).
1576:
1577: \bibitem{gros07} C. Gros,
1578: {\it ``Autonomous Dynamics in Neural networks:
1579: The dHAN Concept and Associative Thought Processes''},
1580: Cooperative Behaviour in Neural Systems (Ninth Granada Lectures),
1581: P.L.~Garrido, J.~Marro, J.J.~Torres (Eds.),
1582: AIP Conference Proceedings {\bf 887}, 129-138;
1583: also available as q-bio.NC/0703002.
1584:
1585: \bibitem{abeles95} M. Abeles {\it et al.},
1586: {\it ``Cortical activity flips among quasi-stationary states''},
1587: Proc. Natl. Acad. Sci. USA {\bf 92}, 8616 (1995).
1588:
1589: \bibitem{kenet03} T. Kenet, D. Bibitchkov, M. Tsodyks, A. Grinvald
1590: and A. Arieli,
1591: {\it ``Spontaneously emerging cortical representations of visual
1592: attributes''},
1593: Nature {\bf 425}, 954 (2003).
1594:
1595: \bibitem{ringach03} D.L. Ringach,
1596: {\it ``States of mind''},
1597: Nature {\bf 425}, 912 (2003).
1598:
1599: \bibitem{hutt03} A. Hutt, H. Riedel,
1600: {\it ``Analysis and modeling of quasi-stationary multivariate
1601: time series and their application to middle
1602: latency auditory evoked potentials''},
1603: Physica D {\bf 177}, 203 (2003).
1604:
1605: \bibitem{freeman03} W.J. Freeman,
1606: {\it ``Evidence from human scalp electroencephalograms of
1607: global chaotic itinerancy''},
1608: Chaos {\bf 13}, 1067 (2003).
1609:
1610: \bibitem{kaneko03} See K. Kaneko, I. Tsuda,
1611: {\it ``Chaotic itinerancy''},
1612: Chaos {\bf 13}, 926 (2003) and references therein.
1613:
1614: \bibitem{timme02} M. Timme, F. Wolf, T. Geisel,
1615: {\it ``Prevalence of Unstable Attractors in Networks of
1616: Pulse-Coupled Oscillators''},
1617: Phys. Rev. Lett. {\bf 89}, 154105 (2002).
1618:
1619: \bibitem{treves05} A. Treves,
1620: {\it ``Frontal latching networks: a possible neural basis for
1621: infinite recursion''}, Cog. Neuropsych. {\bf 22}, 276 (2005).
1622:
1623: \bibitem{kropff06} E. Kropff, A. Treves,
1624: {\it ``The complexity of latching transitions in large scale
1625: cortical networks''}, preprint.
1626:
1627: \bibitem{rabinovich01} M. Rabinovich, A. Volkovskii, P. Lecanda, R. Huerta,
1628: H D.I. Abarbanel, G. Laurent,
1629: {\it ``Dynamical Encoding by Networks of Competing Neuron Groups:
1630: Winnerless Competition''},
1631: Phys. Rev. Lett. {\bf 87}, 068102 (2001).
1632:
1633: \bibitem{seliger03} P. Seliger, L.S. Tsimring, M.I. Rabinovich,
1634: {\it ``Dynamics-based sequential memory: Winnerless competition
1635: of patterns''},
1636: Phys. Rev. E {\bf 67}, 011905 (2003).
1637:
1638: \bibitem{rabinovich06a} M.I. Rabinovich, P. Varona, A.I. Selverston, H.D.I. Abarbanel,
1639: {\it ``Dynamical principles in neuroscience''},
1640: Rev. Mod. Phys. {\bf 78}, 1213 (2006).
1641:
1642: \bibitem{sompolinsky86} H. Sompolinsky and I. Kanter,
1643: {\it ``Temporal Association in Asymmetric Neural Network''},
1644: Phys. Rev. Lett. {\bf 57}, 2861 (1986).
1645:
1646: \bibitem{horn89} D. Horn and M. Usher,
1647: {\it ``Neural networks with dynamical thresholds''},
1648: Phys. Rev. A {\bf 40}, 1036 (1989).
1649:
1650: \bibitem{metzler01} R. Metzler, W. Kinzel, L. Ein-Dor, I. Kanter,
1651: {\it ``Generation of unpredictable time series by a neural network''},
1652: Phys. Rev. E {\bf 63}, 056126 (2001).
1653:
1654: \bibitem{paula06} D.R. Paula, A.D. Ara\'ujo, J.S. Andrade,
1655: H.J. Herrmann, J.A.C. Gallas,
1656: {\it ``Periodic neural activity induced by network complexity''},
1657: Phys. Rev. E {\bf 74}, 017102 (2006).
1658:
1659: \bibitem{arbib02} M.A. Arbib,
1660: {\it ``The Handbook of Brain Theory and Neural Networks''},
1661: MIT Press (2002).
1662:
1663: \bibitem{note_functions} The reservoir functions have
1664: the form of generalize Fermi-functions. A possible
1665: mathematical implementation for
1666: $f_\alpha(\varphi)$, with $\alpha=w,z$,
1667: which we used is
1668: $
1669: f_\alpha(\varphi)\ =\ f_\alpha^{(min)} \,+\,
1670: \left(1.0-f_\alpha^{(min)}\right)
1671: {
1672: {\rm atan}[(\varphi-\varphi_c^{(\alpha)})/\Gamma_\varphi] -
1673: {\rm atan}[(0-\varphi_c^{(\alpha)})/\Gamma_\varphi]
1674: \over
1675: {\rm atan}[(1-\varphi_c^{(\alpha)})/\Gamma_\varphi] -
1676: {\rm atan}[(0-\varphi_c^{(\alpha)})/\Gamma_\varphi]
1677: }
1678: $ with $\varphi_c^{(z)}=0.15$, $\varphi_c^{(w)}=0.7$
1679: $\Gamma_\varphi=0.05$, $f_w^{(min)}=0.1$ and $f_z^{(min)}=0$.
1680:
1681:
1682: \bibitem{chechik01} G. Chechik, I. Meilijson and E. Ruppin,
1683: {\it ``Effective Neuronal Learning with Ineffective
1684: Hebbian Learning Rules''},
1685: Neural Computation {\bf 13}, 817 (2001).
1686:
1687: \bibitem{okada96} M. Okada,
1688: {\it ``Notions of Associative Memory and Sparse Coding''},
1689: Neural Netw. {\bf 9}, 1429 (1996).
1690:
1691: \bibitem{amit85} D.J. Amit, H. Gutfreund, H. Sompolinsky,
1692: {\it ``Storing Infinite Numbers of Patterns in a Spin-Glass Model
1693: of Neural Networks''},
1694: Phys. Rev. Lett. {\bf 55}, 1530 (1985).
1695: %
1696: \bibitem{malsburg86} C. von der Malsburg and W. Schneider,
1697: {\it ``A neural cocktail-party processor''},
1698: Biological Cybernetics, {\bf 54}, 29 (1886).
1699: %
1700: \bibitem{gray89} C.M. Gray, P. K\"onig, A.K. Engel and W. Singer,
1701: {\it ``Oscillatory responses in cat visual cortex exhibit
1702: incolumnar synchronization which reflects global
1703: stimulus properties''},
1704: Nature {\bf 338}, 334 (1989).
1705: %
1706: \bibitem{huerta01} R. Huerta, M. Rabinovich,
1707: {\it ``Reproducible Sequence Generation In Random Neural Ensembles''},
1708: Phys. Rev. Lett. {\bf 93}, 238104 (2004).
1709: %
1710: \bibitem{rabinovich06b} M.I. Rabinovich, R. Huerta, P. Varona and V.S. Afraimovich,
1711: {\it ``Generation and reshaping of sequences in neural systems''}
1712: Biol Cybern. {\bf 95}, 519 (2006).
1713: %
1714: \bibitem{gros05} C. Gros,
1715: {\it ``Self-Sustained Thought Processes in a Dense Associative Network''},
1716: Springer Lecture Notes in Artificial Intelligence (KI2005)
1717: {\bf 3698}, 375 (2005); also available as q-bio.NC/0508032.
1718:
1719: \bibitem{bienenstock82} E.L. Bienenstock, L.N. Cooper, P.W. Munro,
1720: {\it ``Theory for the development of neuron selectivity:
1721: Orientation specificity and binocular interaction in visual cortex''},
1722: J. Neuroscience {\bf 2}, 32 (1982).
1723:
1724: \end{thebibliography}
1725: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
1726:
1727: \end{document}
1728:
1729: