1: \documentclass[]{article}

2: %\documentclass[twocolumn]{article}

3: \addtolength{\hoffset}{-0.6in}

4: \addtolength{\textwidth}{1.2in}

5: \addtolength{\voffset}{-0.6in}

6: \addtolength{\textheight}{1.2in}

7: 

8: %

9: %--useful packages--

10: %\usepackage{doublespace}

11: \usepackage{latexsym}

12: \usepackage{amsmath}

13: \usepackage{amssymb}

14: \usepackage{amsfonts}

15: %\usepackage{times}

16: \usepackage{graphicx}

17: \usepackage{epsfig}

18: \usepackage{array}

19: %\usepackage{flafter}

20: %\usepackage[all]{xy}

21: 

22: %--------------------- Some LaTeX definitions ------------

23: 

24: % CUSTOMIZATIONS

25: \newtheorem{definition}{Definition}

26: \newtheorem{theorem}{Theorem}

27: 

28: \newcommand{\eq}[1]{equation~\ref{eq#1}}

29: \newcommand{\Eq}[2]{

30: \begin{equation}

31: #1

32: \label{eq#2}

33: \end{equation} }

34: 

35: \newcommand{\Eqn}[1]{\[ #1 \]}

36: 

37: \newcommand{\Ex}[1]{Example~\ref{ex#1}}

38: \newcommand{\ex}[1]{example~\ref{ex#1}}

39: 

40: 

41: % ------------------ Defining the \Example environment ---------------------

42: %Syntax:

43: %  \Example{title}{body}{label}

44: %  \Ex{label} ...  will produce Example xxx ..

45: %  ... \ex{label} ..... will produce: ... example xxx ....

46: %  the numbering of the examples (xxx) is done according to the section

47: %  number.

48: 

49: \newtheorem{ExampleDef}{Example}[section]

50: 

51: \newcommand{\Example}[3]{

52:   \begin{list}{}{

53:       \setlength{\leftmargin}{1em}}     % Indent everything by this amount

54:     \item                               % Group everything in one item

55:     \small                              % Use a smaller font size

56:     \begin{ExampleDef} \rm              % Theorems are italic - select roman

57:       {\bf \hspace{-1ex}: #1}           % The name, use \\[1ex] to break line

58:       #2                                % The actual stuff

59:       \hfill {\large \boldmath $\Box$}  % The box

60:       \label{ex:#3}                      % Label the example

61:     \end{ExampleDef}

62:   \end{list}}

63: 

64: %----------------------------------------------------------------------------

65: \setlength{\parskip}{8pt}

66: 

67: 

68: \begin{document}

69: %\singlespace

70: \begin{center}

71: {\Large {\bf Fast Computation with Neural Oscillators \par}}

72: \vspace{1.0em}

73: {\large Wei Wang and Jean-Jacques E. Slotine\footnote{To whom correspondence 

74: should be addressed.} \par} 

75: {Nonlinear Systems Laboratory \\

76: Massachusetts Institute of Technology \\

77: Cambridge, Massachusetts, 02139, USA 

78: \\ wangwei@mit.edu, \ jjs@mit.edu 

79: \par}

80: \vspace{2em}

81: \end{center}

82: 

83: \begin{abstract}

84: Artificial spike-based computation, inspired by models of computations

85: in the central nervous system, may present significant performance

86: advantages over traditional methods for specific types of large scale

87: problems. In this paper, we study new models for two common instances

88: of such computation, winner-take-all and coincidence detection. In

89: both cases, very fast convergence is achieved independent of initial

90: conditions, and network complexity is linear in the number of inputs.

91: \end{abstract}

92: 

93: %\doublespace

94: 

95: \section{Introduction} \label{sec:introduction}

96: 

97: Recent research has explored the notion that artificial spike-based

98: computation, inspired by models of computations in the central nervous

99: system, may present significant advantages for specific types of large

100: scale problems~\cite{hopfield, spiking_sum, binding, jin02, llinas03, lazzaro}.  

101: This intuition is motivated in part by the fact

102: that while neurons in the brain are enormously "slower" than silicon

103: based elements (about six orders of magnitude in both elementary

104: computation time and signal transmission speed), their performance in

105: networks often compares very favorably with their artificial

106: counterparts even when reaction speed is concerned. In a sense,

107: evolution may have been forced to develop extremely efficient

108: computational schemes given available hardware limitations.

109: 

110: In this paper, we study new models for two common instances

111: of such computation, winner-take-all and coincidence detection. In

112: both cases, very fast convergence is achieved and network complexity

113: is linear in the number of inputs. 

114: 

115: We first present a simple network of FitzHugh-Nagumo (FN) neurons for

116: fast winner-take-all computation. In contrast to most existing

117: studies, e.g. the recent~\cite{jin02}, the network's initial state

118: can be arbitrary, and its convergence is guaranteed in at most two

119: spiking periods, making it particularly suitable to track time-varying

120: inputs. If several neurons receive the same largest input, they all

121: spike as a group.

122: 

123: Using a very similar architecture, but replacing global inhibition by

124: global excitation, we obtain an FN network for fast coincidence detection,

125: in a spirit similar to~\cite{hopfield}. Again the system's

126: response is practically immediate, regardless of the number of inputs.

127: 

128: In section 2 we review basic properties of the FitzHugh-Nagumo model

129: in its dimensionless version. Sections 3 and 4 discuss applications to

130: the design of fast winner-take-all networks and coincidence detection

131: networks. Brief concluding remarks are offered in Section 5.

132: 

133: \section{The FitzHugh-Nagumo Model} \label{sec:f-n}

134: 

135: The FitzHugh-Nagumo model~\cite{fitzhugh-nagumo} is a well-known

136: simplified version of the classical Hodgkin-Huxley

137: model~\cite{hodgkin-huxley}, the first mathematical model of wave

138: propagation in squid nerve. Originally derived from the Van der Pol

139: oscillator~\cite{books_vdp}, it can be generalized using

140: a linear state transformation to the dimensionless

141: system~\cite{murray}

142: \begin{equation} \label{eq:f-n}

143: \begin{cases}  

144:   \ \dot{v} = v(\alpha - v)(v-1)-w+I  \\  

145:   \ \dot{w} = \beta v - \gamma w

146: \end{cases}  

147: \end{equation}

148: where $\alpha, \beta, \gamma$ are positive constants. Here $v$ models membrane

149: potential, $w$ accommodation and refractoriness, and $I$ stimulating

150: current.

151: 

152: Simple properties of the FN model~\cite{murray} can be exploited for

153: neural computations. For appropriate parameter choices, there exists

154: a unique equilibrium point for any given value of $I$. Furthermore,

155: this equilibrium point is stable, except for a finite range $\ I_l \le

156: I \le I_h\ $ where the system tends to a limit cycle.  The

157: steady-state value of $v$ at the stable equilibrium point increases

158: with $I$.

159: 

160: %Simple properties of the FN model can be exploited for neural

161: %computations. Consider two null-clines of (\ref{eq:f-n}) in the

162: %phase-plane (Figure~\ref{fig:PhasePlane})

163: %$$

164: %w = v(\alpha - v)(v-1)+I \ \ \ ,  \ \ \

165: %w = \frac{\beta}{\gamma} v   

166: %$$ 

167: %If we set $\ \beta / \gamma \ $ (the slope of the straight

168: %null-cline) large enough, there exists only one equilibrium point for

169: %any value of $I$. This equilibrium point is stable, except for a

170: %finite range $\ I_l \le I \le I_h\ $ where the system tends to a limit

171: %cycle for appropriate parameter choices.  The steady-state value of

172: %$v$ at the stable equilibrium point increases with $I$.

173: %\begin{figure}[h]

174: %\begin{center}

175: %\epsfig{figure=phaseplane.eps,height=50mm,width=100mm}

176: %\caption{ Null-clines of the FitzHugh-Nagumo model~(\ref{eq:f-n}) in

177: %the phase plane. (a) The cubic curve corresponds to $\ \dot{v} =0\ $

178: %with $I = 0$. The two straight lines correspond to $\ \dot{w} =0\ $ with

179: %different values of $\ \beta / \gamma \ $. (b) Fix $\ \beta / \gamma \

180: %$ and vary current input $I$, with $\ 0 < I_1 < I_2 < I_3 \ $.}

181: %\label{fig:PhasePlane}

182: %\end{center}

183: %\end{figure}

184: 

185: %

186: \section{Winner-Take-All Network} \label{sec:wta}

187: %

188: Winner-take-all (WTA) networks, which pick the largest element from a

189: collection of inputs, are ubiquitous in models of neural computation,

190: and have been used extensively in the contexts of competitive

191: learning, pattern recognition, selective visual attention, and

192: decision making \cite{wta_sum, deliang99, yuille02}. Furthermore,

193: Maass~\cite{maass00} showed that WTA represents a powerful computational

194: primitive as compared to standard neural network models based on

195: threshold or sigmoidal gates.

196: 

197: The architectures of most existing WTA models are based on inhibitory

198: interactive networks, implemented either by a global inhibitory unit

199: or by mutual inhibitory connections. Many studies, such

200: as~\cite{old_wta}, require the system dynamics to be

201: initiated from a particular state, which prevents real-time tracking

202: of time-varying inputs. Starting with~\cite{lazzaro}, many

203: WTA implementations in analog VLSI circuits have been

204: proposed. While they do guarantee a unique global minimum, dynamic

205: analysis is difficult and computation resolution limited. Studies of

206: spike-based WTA computation, as in~\cite{jin02}, are comparatively

207: recent.  In this section, we describe a very simple network of

208: FitzHugh-Nagumo neurons for fast winner-take-all computation, whose

209: complexity is linear in the number of inputs. The network's initial

210: state can be arbitrary, and its convergence is guaranteed in at most

211: two spiking periods, making it particularly suitable to track

212: time-varying inputs. If several neurons receive the same largest

213: input, they all spike as a group of winners.

214: 

215: \begin{figure}[h]

216: \begin{center}

217: \epsfig{figure=wta_structure.eps,height=40mm,width=60mm}

218: \caption{Diagram of the network. There are $n$ FN neurons receiving 

219: different external stimulating inputs. A global inhibition neuron

220: monitors the network.}

221: \label{fig:structure}

222: \end{center}

223: \end{figure}

224: As illustrated in Figure~\ref{fig:structure}, the network consists of 

225: $n$ FN neurons. Each neuron receives a stimulating input $I_i$ from outside 

226: as well as a common inhibition current $z$ from the global inhibition 

227: neuron. The dynamics of the FN neurons ($i=1,\ldots,n$) are

228: \begin{equation*} \label{eq:fn-in-wta}

229: \begin{cases}  

230:   \ \dot{v}_i = v_i (\alpha - v_i) (v_i-1)-w_i + I_i - z  \\  

231:   \ \dot{w}_i = \beta v_i - \gamma w_i

232: \end{cases}  

233: \end{equation*}

234: The dynamics of the global inhibition neuron switches between a

235: charging mode and a discharging mode. It starts charging if there is

236: any FN neuron spiking in the network, i.e. when $v_i$ exceeds a

237: given threshold value $v_0$.  It switches to discharging if the state

238: is saturated (enough close to the saturation value in simulation)

239: and stays at this mode until next time a FN neuron

240: spikes. The specific dynamics of these two modes can be very

241: general. For simplicity, we use

242: \begin{equation*} \label{eq:inhibition-in-wta}

243: \dot{z} = 

244: \begin{cases}  

245:   \  - k_c \ (z - z_0)  \ \ \ \ \mathrm{charging}\ \mathrm{mode} \\  

246:   \  - k_d \ z   \ \ \ \ \ \ \ \ \ \ \ \ \mathrm{discharging}\ \mathrm{mode}

247: \end{cases}  

248: \end{equation*}

249: where $z_0$ is a constant saturation value, and $k_c$ and $k_d$ are the

250: charging rate and discharging rate.

251: 

252: Within at most two periods, the winner will be the only neuron

253: spiking in the whole network. In some cases, this is achieved

254: within the first period. 

255: 

256: To perform WTA computation, we set the charging rate of the global

257: neuron to be fast and the discharging rate to be slow. Thus, similarly

258: to~\cite{jin02}, if there is any FN neuron spiking, the strength of

259: the inhibition current increases to its saturation value very rapidly,

260: leaving no chance for the other neurons to spike. The global neuron

261: then discharges slowly, which lets the FN neurons smoothly approach

262: the oscillation region. The first neuron entering the oscillation

263: region will be the one with the largest input. So it spikes as the

264: winner and ignites a new period. Note that before enter the

265: oscillation region, all FN neurons converge to their equilibrium

266: points with the equilibrium point of the winner having the largest

267: value. A slowly discharging process allows the winner to occupy the

268: highest position and helps it to spike immediately once it enters the

269: oscillation region.  Given the parameters of the FN neurons, the

270: frequency of the result depends on the global neuron's saturation

271: value, its charging and discharging rates, and the value of the

272: largest input. If we also fix the global neuron dynamics, the

273: frequency increases with the increasing of the largest input.

274: Simulation results are shown in Figure~\ref{fig:wta-normal}.

275: \begin{figure}[h]

276: \begin{center}

277: \epsfig{figure=wta_normal.eps,height=70mm,width=80mm}

278: \caption{ Simulation result of WTA computation with $n=10$.  The

279: parameters of the FN neurons are set as $\alpha = 5.32, \beta = 3,

280: \gamma = 0.1$, with spiking threshold $\ v_0 = 5\ $.  The inputs $I_i$

281: are chosen randomly from $20$ to $125$. The parameters of the global

282: neuron are $z_0 = 160, k_c = 1, k_d = 1/50$. All the initial

283: conditions are chosen arbitrarily. (a) States $v_i$ versus time. The

284: dashed curve represents the state of the neuron receiving the largest

285: input. (b) State $z$ versus time.}

286: \label{fig:wta-normal}

287: \end{center}

288: \end{figure}

289: 

290: \noindent {\em {\large Remarks}}

291: 

292: \noindent $\bullet$ {\bf Initial conditions and computation speed}\ \

293: \ \ \ The mechanism described above guarantees that initial conditions

294: can be set arbitrarily, which cannot be realized by most of the

295: previous WTA models. With appropriate parameters, the computation can

296: be completed at most in two periods. The first spiking neuron is

297: chosen by initial conditions, while the second one is the neuron with

298: the largest input, which remains the winner until the inputs change.

299: Actually, if the initial inhibition is set large enough so that all

300: the FN neurons are depressed in the beginning, then the neuron with

301: the largest input will spike first. The computation speed of our FN

302: network is faster and more robust than the WTA model recently

303: presented in~\cite{jin02}, whose network of integrate-and-fire neurons

304: has to wait until the winner gets the right to spike, which may take a

305: long time for large networks.

306: 

307: \noindent $\bullet$ {\bf Varying inputs and noise} \ \ \ \ \ Since

308: initial conditions do not matter in our model, the network can easily

309: track time-varying inputs. Figure~\ref{fig:wta-varying} illustrates

310: such an example, where three inputs switch winning positions several

311: times. The spiking neuron always tracks the largest input. The

312: computation is robust to signal noise as well.

313: \begin{figure}[h]

314: \begin{center}

315: \epsfig{figure=wta_varying.eps,height=70mm,width=80mm}

316: \caption{ Simulation result of WTA computation with varying input. The

317: parameters are all equal to those in Figure~\ref{fig:wta-normal}

318: except that $n=3$. Inputs change continuously. The winner always

319: tracks the neuron with the largest input.}

320: \label{fig:wta-varying}

321: \end{center}

322: \end{figure}

323: 

324: \noindent $\bullet$ {\bf Multiple winners} \ \ \ \ Decreasing the

325: global neuron discharging rate $k_d$ extends the waiting time before

326: the winner enters the oscillation region. This is helpful if there

327: exist several neurons receiving the same largest input and we expect

328: them all spike as a group of winners. Enlarging the time neurons stay

329: in the stable region allows these neurons with the same input converge

330: to each other, and to enter the oscillation region and spike

331: simultaneously ( Figure~\ref{fig:wta-multiple}).  Note that

332: in~\cite{jin02}, only one winner can succeed and it is picked

333: arbitrarily from the group of candidates.

334: 

335: If the network size is small, the network may be augmented with

336: all-to-all couplings {\it between} FN neurons, with the coupling gain

337: increasing with the similarity of the inputs (e.g., of the form

338: $e^{- \ \alpha |\ I_j - I_i \ |}$ ). This lets the neurons

339: receiving identical inputs converge together exponentially (using

340: partial contraction theory \cite{wei}) and thus provides

341: another solution to the multiple-winner problem.

342: \begin{figure}[h]

343: \begin{center}

344: \epsfig{figure=wta_multiple.eps,height=40mm,width=70mm}

345: \ \ \ \ \ \ \ \ \epsfig{figure=wta_resolution.eps,height=40mm,width=20mm}

346: \caption{Simulation result of WTA computation with multiple

347: winners. The parameters are the same as in Figure~\ref{fig:wta-normal}

348: except that $k_c = 5,\ k_d = 1/80$.  The inputs are $\ I_1 = \cdots =

349: I_9 = 120, I_{10} = 119.5\ $. The first plot shows the $v_i$'s as

350: functions of time. The dashed (red) line represents $v_{10}$ and the

351: solid (blue) lines the other $v_i$'s. The first nine neurons converge together

352: during the waiting time and spike simultaneously as a group of

353: winners. The second plot, an enlarged version of the first at a

354: spiking moment, shows that $v_{10}$ is completely depressed by the

355: winners even though the input difference is very small.}

356: \label{fig:wta-multiple}

357: \end{center}

358: \end{figure}

359: 

360: \noindent $\bullet$ {\bf Computation resolution} \ \ \ \ Computation

361: resolution can be improved by decreasing the global neuron discharging

362: rate $k_d$ while increasing the charging rate $k_c$. Decreasing $k_d$

363: allows the winner fully distinguished with the following neurons;

364: increasing $k_c$ prevents the following neurons spike after the

365: winner. Figure~\ref{fig:wta-multiple} illustrates such an example with

366: winners $I_{max} = 120$ while the second largest input $I_{10} =

367: 119.5$. The resolution here is much better than the WTA models

368: presented previously, including~\cite{lazzaro,urahama}. It can be

369: further enhanced by decreasing the relaxation time of the FN neurons.

370: 

371: \noindent $\bullet$ {\bf Input bounds} \ \ \ \ The inputs to the FN

372: neurons should be lower-bounded by $I_l$ (the lower threshold of the

373: oscillation region) to guarantee that the neurons can spike before the

374: inhibition is fully released. They should also be upper bounded to set

375: $z_0$.

376: 

377: \noindent $\bullet$ {\bf $K$-Winner-Take-All} \ \ \ \

378: $K$-Winner-Take-All ($k$-WTA) is a common variation of WTA

379: computation~\cite{maass00}, where the output indicates for

380: each neuron whether its input is among the $k$ largest.  An example

381: $k$-WTA circuit in~\cite{urahama} extends the WTA model

382: in~\cite{lazzaro} by formulating the problem in terms of mathematical

383: programming, but it inherits its low resolution limit

384: from~\cite{lazzaro} as well. Conversely, the advantages of the above

385: FN network generalize to the $k$-WTA case.  As inhibition decreases,

386: the FN neurons enter the oscillation region rank-ordered by their

387: inputs. For WTA computation, we charge the inhibition neuron after the

388: first arrival. For $k$-WTA computation, we only need to modify the

389: charging moment to capture the $k^{\rm th}$ arrival instead. Since the

390: neurons enter the oscillation region in sequence, they spike in

391: sequence. If we set $k=n$, we get a pre-ordered spiking sequence in

392: each period, which may be used to realize soft-WTA~\cite{maass00,

393: yuille02}, and also provides a simple desynchronization mechanism for

394: binding problems~\cite{binding}. The

395: computation resolution follows directly from that in WTA.  A detailed

396: description and discussion of the $k$-WTA network will be presented

397: separately.

398: 

399: \noindent $\bullet$ {\bf Spike-controlled coupling and slow

400: inhibition} \ \ \ \ The feedforward and recurrent connections used in

401: our WTA network are similar to those in~\cite{llinas03}, where a

402: ``universal'' control system is developed based on olivo-cerebellar

403: networks. The couplings inside the circuit are also spike-controlled

404: and they use a FitzHugh-Nagumo-like model containing four variables. A

405: similar mechanism is also used in~\cite{deliang99}, where WTA is

406: implemented to compute the object with the largest

407: size. Slowly-discharged inhibition is also used in biologically

408: motivated models such as~\cite{slow_inhinition, deliang99}.

409: 

410: \noindent $\bullet$ {\bf Computational complexity} \ \ \ \ \ 

411: The complexity of the network is $O(n)$. Since the FN neurons are

412: independent, they can be added or removed from the network at any

413: time.

414: 

415: %

416: \section{Fast Coincidence Detection} \label{sec:fsd}

417: %

418: Recent neuroscience research suggests that coincidence detection plays

419: a key role in temporal binding~\cite{coincidence}. Hopfield {\em

420: et al.}~\cite{hopfield} proposed two neural network

421: structures, both able to capture a ``many-are-equal'' moment, to model

422: speech recognition and olfactory processing. A similar computation can

423: be implemented by FN neurons, with faster and more salient response.

424: 

425: Consider a leader-followers network with a structure similar to

426: Figure~\ref{fig:structure}, except that the global neuron (the leader)

427: is now {\it excitatory}, and the connections from the leader to the

428: followers are unidirectional. For simplicity, we assume that all the

429: neurons are FN neurons with the same parameters but different

430: inputs. The dynamics of the leader $(v_o , w_o)$ obeys

431: equations~(\ref{eq:f-n}) while those of the followers ($i=1,\ldots,n$)

432: are

433: \begin{equation*}

434: \begin{cases}  

435:   \ \dot{v}_i = v_i(\alpha - v_i)(v_i-1)-w_i+I_i + k(v_0-v_i)  \\  

436:   \ \dot{w}_i = \beta v_i - \gamma w_i

437: \end{cases}  

438: \end{equation*}

439: where $k(v_0-v_i)$ is the coupling force from the leader to the

440: followers.  Neurons $i$ and $j$ synchronize only if inputs $I_i$ and

441: $I_j$ are identical.  We define the system output accordingly to

442: capture the moment when this condition becomes true for a large number

443: of inputs, as illustrated in Figure~\ref{fig:hopfield}. Note that the

444: coupling gain $k$ should be large enough to guarantee synchronization

445: (an explicit threshold can be computed

446: analytically~\cite{wei}), but not so large as to have the

447: leader numerically dominate the dynamic differences between the

448: followers. More general formal studies of synchronization can be found

449: in~\cite{wei}, based on nonlinear contraction theory.

450: 

451: \begin{figure}[h]

452: \begin{center}

453: \epsfig{figure=hopfield.eps,height=70mm,width=100mm}

454: \caption{Simulation result of fast coincidence detection with

455: $n=30$. The leader and the followers are FN neurons with the same

456: parameters as in Figure~\ref{fig:wta-normal}. The inputs are $I_0=90$

457: and $I_1, \ldots, I_{30}$ varying from $20$ to $80$, and the coupling

458: gain is $k=1.7$. The upper plot shows $\ \sum_{i=1}^{n} \max (0,

459: \dot{v_i})$ versus time, and the lower $I_1, \ldots, I_n$.}

460: \label{fig:hopfield}

461: \end{center}

462: \end{figure}

463: 

464: %

465: \section{Concluding Remarks} \label{sec:conclusion}

466: %

467: Basic computations such as winner-take-all and coincidence detection

468: can be performed fast and robustly using extremely simple spike-based

469: models. The results are currently being extended to higher-level

470: perception problems.

471: 

472: \vspace{1.5em}

473: 

474: \noindent {\large{\bf Acknowledgments:}} This work was supported in

475: part by a grant from the National Institutes of Health. The authors

476: benefited from stimulating discussions with Matthew Tresch.

477: 

478: %\input{ref_short.tex}

479: 

480: %

481: %reference

482: %

483: %\begin{thebibliography}{XX}

484: \renewcommand{\baselinestretch}{1.}

485: \begin{thebibliography}{0}

486: \setlength{\parskip}{.2ex}

487: \setlength{\itemsep}{.2ex}

488: \footnotesize

489: 

490: \bibitem{wta_sum} Arbib, M.A., {\em MIT Press} (1995);

491: Amari, S., and Arbib, M.,

492: in {\em Sys. Neurosci.}, 119 (1977);

493: Ermentrout, B.,

494: {\em Neural Networks}, {\bf 5}(3) (1992);

495: Fang, Y., Cohen, M., and Kincaid, M.,

496: {\em Neural Networks}, {\bf 9}:1141 (1996);

497: Grossberg, S.,

498: {\em Stud. in Appl. Math.}, {\bf 52}:217 (1973);

499: Grossberg, S., 

500: {\em J. of Math. Analy. and Appl.}, {\em 66}:470 (1978);

501: Indiveri, G.,

502: {\em IEEE Internat. Joint Conf. on Neural Networks}, {\bf 4}:24 (2000)

503: 

504: 

505: \bibitem{hopfield} Brody, C.D., and Hopfield, J.J.,

506: {\em Neuron}, {\bf 37}:843  (2003);

507: Hopfield, J.J., and Brody, C.D.,

508: {\em Proc. Natl. Acad. Sci. USA}, {\bf 98}:1282 (2001)

509: 

510: 

511: \bibitem{spiking_sum} Dehaene, S., {\em et al.},

512: {\em PNAS} {\bf 95}:14529 (1998);

513: Gerstner, W.,

514: In {\em The Handbook of Bi. Phy.}, {\bf 4}:469  (2001);

515: Izhikevich, E.M.,

516: {\em IEEE Trans. on Neural Networks}, (2003);

517: Llin\'{a}s, {\em et al.},

518: {\em Phil. Trans. R. Soc. Lond.} {\bf 353}:1841  (1998);

519: Maass, W., Natschl$\mathrm{\ddot{a}}$ger, T., and Markram, H. (2003); 

520: Thorpe, S., {\em et al.},

521: {\em Nature}, {\bf 381}:520 (1996);

522: Thorpe, S., {\em et al.},

523: {\em Neural Networks}, {\bf 14}:715 (2001)

524: 

525: 

526: \bibitem{old_wta} Feldman, J.A., and Ballard, D.H.,

527: {\em Cognit. Sci.}, {\bf 6}:205 (1982);

528: Yuille, A.L., and Grzywacz, N.M.,

529: {\em Neural Comput.}, {\bf 1}:334  (1989)

530: 

531: \bibitem{fitzhugh-nagumo} FitzHugh, R.A.,

532: {\em Biophys. J.}, {\bf 1}:445 (1961);

533: Nagumo, J., {\em et al.}, 

534: {\em Proc. Inst. Radio Engineers}, {\bf 50}:2061 (1962)

535: 

536: 

537: \bibitem{binding} Gray, C.M., 

538: {\em Neuron}, {\bf 24}:31 (1999);

539: Singer, W., and Gray, C.M.,

540: {\em Annu. Rev. Neurosci.}, {\bf 18}:555 (1995);

541: von der Malsburg, C.,

542: {\em Current Opinion in Neurobio.}, {\bf 5}:520 (1995);

543: Wang, D.L.,

544: The Time Dimension for Neural Computation, (2002) 

545: 

546: 

547: \bibitem{hodgkin-huxley} Hodgkin, A.L., and Huxley, A.F.,

548: {\em J. Physiol.,} {\bf 117}:500 (1952)

549: 

550: 

551: \bibitem{jin02} Jin, D.Z., and Seung, H.S.,

552: {\em Phy. Rev. E}, {\bf 65}:051922 (2002)

553: 

554: \bibitem{llinas03} Kazantsev, V.B., {\em et al.},

555: {\em PNAS}, {\bf 100}:13064 (2003)

556: 

557: 

558: \bibitem{slow_inhinition} LoFaro, T., {\em et al.},

559: {\em Neural Comput.}, {\bf 6}:69 (1994);

560: Wang, X.J., and Rinzel, J.,

561: {\em Neural Comput.}, {\bf 4}:84 (1992)

562: 

563: \bibitem{lazzaro} Lazzaro, J., {\em et al.},

564: in {\em Advan. in Neural Info. Proc. Sys. 1.} 703 (1988)

565: 

566: \bibitem{coincidence} Llin\'{a}s,  {\em et al.},

567: {\em PNAS}, {\bf 99}:449 (2002);

568: Tononi, G., {\em et al.},

569: {\em PNAS} {\bf 95}:3198 (1998)

570: 

571: 

572: \bibitem{maass00} Maass, W.,

573: {\em Neural Comput.}, {\bf 12}:2519 (2000);

574: Maass, W.,

575: in {\em Advan. in Info. Proc. Sys.}, {\bf 12}:293 (2000)

576: 

577: \bibitem{murray} Murray, J.D.,

578: Mathematical Biology, (1993)

579: 

580: \bibitem{books_vdp} Slotine, J.J.E., and Li, W.,

581: Applied Nonlinear Control, (1991);

582: Strogatz, S.H.,

583: Nonlinear Dynamics and Chaos, (1994)

584: 

585: \bibitem{wei} Slotine, J.J.E., and Wang, W.,

586: in {\em Cooper. Contr.}, Springer-Verlag, (2003);

587: Wang, W., and Slotine, J.J.E.,

588: {\em Biol. Cybern.}, submitted (2003)

589: 

590: \bibitem{urahama} Urahama, K., and Nagao, T.,

591: {\em IEEE Trans. on Neural Networks},{\bf 6}:776 (1995)

592: 

593: \bibitem{deliang99} Wang, D.L.,

594: {\em Neural Networks}, {\bf 12}:579 (1999)

595: 

596: \bibitem{yuille02} Yuille, A.L., and Geiger, D.,

597: in {\em The Handbook of Brain Theory and Neural Networks}, (2002)

598: 

599: \end{thebibliography}

600: 

601: \end{document}

602: 

603: 

604: 

605: 

606: 

607: 

608: 

609: