1: % Persistent Chaos in High Dimensions

2: % da:  12/9/04

3: % jpc: 12/13/04, 3/25/05, 9/11/05, 10/04/05

4: % da: 9/23/05

5: 

6: \documentclass[superscriptaddress,twocolumn,floatfix,showkeys,preprintnumbers,showpacs]{revtex4}

7: \usepackage{amscd}

8: \usepackage{epsfig}

9: \usepackage{amsmath}

10: \usepackage{amssymb}

11: 

12: \renewcommand{\textfraction}{.01}

13: \renewcommand{\topfraction}{.99}

14: \renewcommand{\bottomfraction}{.99}

15: 

16: \newtheorem{definition}{Definition}

17: \newtheorem{lemma}{Lemma}

18: \newtheorem{theorem}{Theorem}

19: \newtheorem{corollary}{Corollary}

20: \newtheorem{proposition}{Proposition}

21: \newtheorem{conjecture}{Conjecture}

22: \newtheorem{assumption}{Assumption}

23: \newtheorem{condition}{Condition}

24: \newtheorem{sor}{Statement of Results}

25: 

26: \def\TEXPATH{./}

27: 

28: \begin{document}

29: \bibliographystyle{plain}

30: 

31: \title{Persistent Chaos in High Dimensions}

32: \author{D. J. Albers}

33: \email{albers@cse.ucdavis.edu}

34: \affiliation{Max Plank Institute for Mathematics in the Sciences,

35:   Leipzig 04103, Germany}

36: \affiliation{Computational Science and Engineering Center and Physics Department,

37: University of California, Davis, One Shields Ave, Davis CA 95616}

38: \affiliation{Physics Department, University of Wisconsin, Madison, WI 53706}

39: \affiliation{Santa Fe Institute, 1399 Hyde Park Road, Santa Fe, NM 87501}

40: 

41: \author{J. C. Sprott}

42: \email{sprott@physics.wisc.edu}

43: \affiliation{Physics Department, University of Wisconsin, Madison, WI 53706}

44: 

45: \author{J. P. Crutchfield}

46: \email{chaos@cse.ucdavis.edu}

47: \affiliation{Computational Science and Engineering Center and Physics Department,

48: University of California, Davis, One Shields Ave, Davis CA 95616}

49: \affiliation{Santa Fe Institute, 1399 Hyde Park Road, Santa Fe, NM 87501}

50: 

51: \date{\today}

52: 

53: \begin{abstract}

54: An extensive statistical survey of universal approximators shows that

55: as the dimension of a typical dissipative dynamical system is

56: increased, the number of positive Lyapunov exponents increases monotonically

57: and the number of parameter windows with periodic behavior decreases. A subset

58: of parameter space remains in which topological change induced by small

59: parameter variation is very common. It turns out, however, that if the system's

60: dimension is sufficiently high, this inevitable, and expected, topological

61: change is never catastrophic, in the sense chaotic behavior is preserved.

62: One concludes that deterministic chaos is persistent in high dimensions.

63: \end{abstract}

64: 

65: \keywords{Chaos, high dimensions, dynamical systems, structural stability,

66:   Lyapunov exponents}

67: \pacs{05.45.-a, 89.75.-k, 05.45.Tp, 89.70.+c, 89.20.Ff}

68: \preprint{Santa Fe Institute Working Paper 05-04-011}

69: \preprint{arxiv.org/abs/nlin/0504040}

70: 

71: \maketitle

72: 

73: %\vspace{-0.5in}

74: 

75: Physical theory attempts to describe and predict the natural world by

76: expressing observed behavior and the governing balance of forces formally

77: in mathematical models---models that can only be approximate representations.

78: Empirically, many natural phenomena persist even when control parameters

79: and external conditions vary. For example, the essential character of fully

80: developed fluid turbulence is little affected if one slightly changes the

81: energy flux that drives it or if a small dent is made in the containing

82: vessel's wall. In building a theory of a system exhibiting this kind of

83: dynamical persistence, one hopes that, despite its approximations,

84: one's model also has this persistence.

85: 

86: \vspace{-0.025in}

87: 

88: A century of analyzing nonlinear dynamical systems, however, has lead to an

89: apparent inconsistency with this goal. Since the days of Poincar\'e's

90: development of qualitative dynamics, mathematicians and physicists have

91: probed differential equations to test their solutions for different kinds of

92: stability. Poincar\'e's discovery of deterministic chaos \cite{poincare_thesis}

93: demonstrated that at the most detailed level, there was inherent instability of

94: system solutions: change the initial condition only slightly and one finds

95: a different state-space trajectory develops rapidly. Later studies showed

96: that there was also an instability in behavior if the equations or parameters

97: were changed only slightly \cite{newhousewild1,Farm85a}. Even arbitrarily

98: small functional perturbations to the governing dynamic leads to radical

99: changes in behavior---from unpredictable to predictable behavior, for example.

100: The overall conclusion has been that nonlinear, chaotic systems are exquisitely

101: sensitive, amplifying arbitrarily small variations in initial and boundary

102: conditions and parameters to macroscopic scales.

103: 

104: %\vspace{-0.07in}

105: 

106: How can one reconcile this with the observed fact of dynamical persistence

107: in many large-scale systems? We take \emph{dynamical persistence} to mean

108: that a behavior type ---e.g., equilibrium, oscillation, chaos---does

109: \emph{not} change with functional perturbation or parameter variation.

110: Here we describe the results of a Monte Carlo survey which empirically

111: demonstrate that chaos is dynamically persistent if the dimension of a

112: nonlinear system is sufficiently high. More constructively, we argue that

113: a particular mechanism is responsible for persistent chaos.

114: 

115: \vspace{-0.03in}

116: 

117: Specifically, the survey shows that in large-scale systems dynamical

118: sensitivity---when defined as breaking topological equivalences associated

119: with structural stability \cite{maness}, ergodicity \cite{pughshubAMS}, and

120: statistical stability \cite{alves_viana_stat_stab}---is typically benign and

121: does not affect behavior types. Naturally, drastic changes in the invariant

122: measure yield different observed dynamics, but the results indicate that

123: this becomes increasingly less probable. Moreover, the instability

124: associated with deterministic chaos dominates high-dimensional dynamical

125: systems, except at extreme parameter settings.

126: 

127: Much of the

128: intuition and motivation for our investigation comes from the analytical

129: results found in abstract dynamical systems theory, but our construction and

130: conclusions highlight a distinct difference. Said most simply, the number of

131: dimensions of the dynamical system matters. That is, there is a qualitative

132: difference between common behaviors in high- and low-dimensional dynamical

133: systems. Beyond giving empirical evidence to support these conclusions, we

134: introduce a definition of persistent chaos that suggests an alternative

135: approach to the long-standing questions of dynamical stability and offer a

136: mathematical conjecture on the mechanism underlying persistent chaos in

137: high dimensions.

138: 

139: \vspace{-0.025in}

140: 

141: The spectrum of Lyapunov characteristic exponents (LCE) \cite{benn2} will be

142: our primary tool for analyzing and identifying behavior types since there

143: is an equivalence between the number of negative and positive Lyapunov

144: exponents and the number of global stable and unstable manifolds,

145: respectively---structures that organize the state space and constrain

146: behavior \cite{ruellehilbert}. Therefore, in referring to topological

147: variation here we mean a change in the number of positive Lyapunov exponents.

148: 

149: \vspace{-0.02in}

150: 

151: In order to give a complete representation of the space of all systems, we

152: investigate typical behaviors in high dimensions using a class of dynamical

153: systems that are known to be \emph{universal function approximators}.

154: (They are universal in that they approximate arbitrarily closely any $C^r$

155: mapping, cf. \cite{hor2}.) These are single-layer neural networks of the form

156: \vspace{-0.1in}

157: \begin{equation}

158: x_{t} = \beta_0

159:       + \sum_{i=1}^{n}{{\beta}_i \tanh s \left( {\omega}_{i0}

160:           + \sum_{j=1}^{d}{{\omega}_{ij} x_{t-j} } \right)} ~,

161: \label{equation:net1}

162: \end{equation}

163: \vspace{-0.03in}

164: which are maps from $R^{d}$ to $R$. Here $n$ is the number of hidden units

165: (neurons), $d$ the number of time lags which determines the system's input

166: (embedding) dimension, and $s$ a scaling factor for the connection weights

167: $w_{ij}$. The initial condition is $({x}_{1}, x_{2}, \ldots, x_{d})$ and the

168: state at time $t$ is $({x}_{t}, x_{t+1}, \ldots, x_{t+d-1})$. The approximation

169: theorems of Ref. \cite{hor2} and time-series embedding results

170: of Ref. \cite{embedology} establish an equivalence between these neural

171: networks and general dynamical systems; cf. \cite{hypviolation}.

172: 

173: \vspace{-0.02in}

174: 

175: In the Monte Carlo survey we sample the $(n(d+1)+1)$-dimensional parameter

176: space taking (i) $\beta_{i} \in [0,1]$ uniformly distributed and rescaled

177: to satisfy $\sum_{i=1}^{n}{{{\beta}_{i}^{2}}} = n$, (ii) ${w}_{ij}$ as

178: normally distributed with zero mean and unit variance (which is adjusted

179: with $s$), and (iii) the initial $x_j \in [-1,1]$ as uniform. These

180: distributions---denoted $m_{\beta}$, $m_{w}(s)$, and $m_I$---form a product

181: measure on the space of parameters and initial conditions---the survey's

182: results then are statistical estimates with respect to this product measure.

183: We use the $s$ parameter as the primary control as it gives the magnitude

184: of the argument of $\tanh(x)$.  When $x \approx 0$ Eq. (\ref{equation:net1})

185: is linear and one finds fixed points and limit cycles; when $|x| \gg 1$

186: the output is binary and one finds $2^n$ different periodic states;

187: and for $|x|$ between these extremes we find the nonlinear behavior

188: we will focus on.

189: 

190: %We will

191: %focus largely on behavior types as a function of the parameter $s$, which

192: %can be interpreted as the standard deviation of the $w$ weight matrix,

193: %and the embedding dimension $d$.

194: 

195: %\vspace{-0.025in}

196: 

197: %Variation in the argument of $\tanh(x)$ leads to different qualitative

198: %behavior types. On the one hand, when $x \approx 0$ it is linear and

199: %one finds fixed points and limit cycles. On the other, when $|x| \gg 1$

200: %the output is binary and one finds $2^n$ different periodic states. $s$

201: %controls moving between these extremes and so it provides a unique

202: %bifurcation parameter. We focus here on $0 < s < 10$---the chaotic region.   

203: 

204: %The \emph{squashing function} $\tanh(x)$, for $|x| \gg 1$, behaves like a binary

205: %function. The neural network states will tend toward the finite set

206: %$x_t = \beta_0 \pm \beta_1 \dots \pm \beta_n$; that is, each $x_t$ can have

207: %$2^n$ different states. The result is that in the limit where the arguments

208: %of $\tanh(\cdot)$ become infinite, the neural network has periodic behavior.

209: %For $x \approx 0$, however, $\tanh(x)$ is nearly linear. Thus,

210: %choosing $s$ to be small forces the dynamics to be mostly linear, again

211: %yielding fixed-point and periodic behavior.  Due to this, $s$

212: %provides a unique bifurcation parameter that sweeps from linear to highly

213: %nonlinear parameter regimes: from discretized binary behavior---fixed

214: %points---to chaos and back to periodic phenomena.   

215: 

216: \vspace{-0.025in}

217: 

218: We define \emph{persistent chaos} in terms of the LCE spectrum as follows:

219: \vspace{-0.09in}

220: \begin{definition}[Degree-$p$ Persistent Chaos]

221: \label{definition:robustchaos}

222: Assume a discrete-time map $f$ that takes a compact set to itself. The map

223: has persistent chaos of degree-$p$ if there exists an open subset $U$

224: of parameter space, such that, for all $\xi \in U$ and a given open set

225: $\mathcal{O}$ of initial conditions, $f|_\xi$ retains $p \geq 1$ positive

226: Lyapunov exponents.

227: \end{definition}

228: \vspace{-0.09in}

229: Persistent chaos (\emph{$p$-chaos}) of degree $p$ is our notion of dynamical

230: equivalence on an open set of parameter space. This differs from that of a

231: robust chaotic attractor of Refs. \cite{unimodalrobust} and

232: \cite{yorkerobustchaos}, for example, in that we do not require the attractor

233: to be unique on the subset $U$. This is an important distinction since,

234: physically, there is little evidence indicating that such strict forms of

235: uniqueness are present in many complex physical systems \cite{milnorkaneko}

236: and, technically, uniqueness is significantly more difficult to demonstrate.

237: Indeed, alternative dynamical equivalence frameworks---such as nonuniform

238: partial hyperbolicity \cite{pesinlebook}---were invented to circumvent

239: problems with nonuniqueness.

240: 

241: \vspace{-0.025in}

242: 

243: \begin{figure}[tbp]

244: \begin{center}

245: \epsfig{file=Figure1.eps, height=5.0cm}

246: \end{center}

247: \caption{LCE spectrum as a function of scale factor $s$ for a network of

248:   $32$ neurons and $64$ dimensions.  ($15000$ total time-steps; $5000$

249:   initial time-steps to arrive on the attractor.)}

250: \label{fig:biglced64}

251: \end{figure}

252: %location:

253: %/Users/albers/research/projects/structuralstability/data/coyote/dataforSSandHypPaper/oct03/n32d64/1/lcecase1n32d64.ps

254: 

255: \vspace{-0.02in}

256: 

257: Figure \ref{fig:biglced64} represents the typical scenario for the LCE

258: spectra\footnote{For a $d$-dimensional system the spectrum consists of $d$

259: LCEs: $\chi_1 \geq \chi_2 \geq \ldots \geq \chi_d$, where indexing gives

260: a monotonic ordering.} of the high-dimensional neural networks as a function

261: of $s$. Here \emph{typical} refers to what was observed in $> 99 \%$ of

262: the $15800$ networks with $n\geq 32$ and $d>32$ given $m_{\beta}$, $m_{w}(s)$, and

263: $m_I$ as defined earlier.  Important features to notice include the lack of

264: periodic windows and that the LCEs vary continuously with $s$ and have a single

265: maximum (up to numerical fluctuations). The survey also reveals that

266: the maximum number of positive Lyapunov exponents is approximately $d/4$

267: and the attractors' Kaplan-Yorke dimension is roughly $d/2$ \cite{mythesis}.

268: As $d$ increases, the length of the $s$-intervals between LCE zero-crossings

269: decreases as $\sim d^{-1.92}$. These properties contrast sharply with the

270: familiar low-dimensional scenarios where one typically

271: encounters a preponderance of stable behavior and periodic windows and the

272: LCEs vary in a discontinuous manner with control parameters. (A more complete

273: analysis of these observations is found in Ref. \cite{hypviolation}.)

274: Moreover, this scenario is different from low-entropy, spatially-extended

275: systems discussed in \cite{grassberger_predictibility} or

276: \cite{kaneko_map_lattice_book}.  In our systems, the effective

277: dimension is irreducibly high and the uniformity of the dynamics is more robust.

278: 

279: \vspace{-0.025in}

280: 

281: These observations complement those from a previous study of chaos in

282: neural-network continuous-time differential equations \cite{Sompolinsky_nn}.

283: There, a mean-field analysis, which assumes that inputs are statistically

284: independent (and which does not apply in the present case), also suggested

285: that chaos should be common in high dimensions. 

286: 

287: \vspace{-0.025in}

288: 

289: In light of Fig. \ref{fig:biglced64} and the fact that LCE zero crossings

290: become asymptotically dense ($|U_i|\sim d^{-1.92}$ for $i<20, d\leqq 128$)

291: we propose the following dynamical mechanism for persistent chaos. For a

292: finite but arbitrarily large number of dimensions along an $s$

293: interval---e.g., $s \in (2, 8)$---there is an asymptotically dense, always

294: countable sequence of parameter values that have an LCE transversally

295: crossing through zero.  Thus, a continuous path along an $s$-interval

296: yields inevitable, but noncatastrophic (i.e. $p > 1$) topological change.

297: This implies that when varying parameters, periodic and quasiperiodic

298: windows will not exist in chaotic regions of parameter space of dynamical

299: systems with a sufficiently large number of positive exponents (i.e., when

300: entropy rate is large). The lack of dense periodic and quasiperiodic windows

301: is a necessary condition for $p$-chaos.

302: 

303: %\vspace{-0.025in}

304: 

305: %The claim is that high-dimensions provide a mechanism not afforded

306: %low-$d$ dynamical systems, one which increases $p$ with $d$ and,

307: %more to the point, increases dynamical persistence. In support of

308: %this, we examine two sets of evidence, the first from along an

309: %$s$-interval and the second in the full $(n(d+1)+1)$-dimensional

310: %parameter space.

311: 

312: \vspace{-0.025in}

313: 

314: To test this picture, we analyzed the existence of periodic and quasiperiodic

315: windows along $s \in (1, 4)$ in networks with $n=32$ and $d$ ranging from

316: $8$ to $128$ and with an ensemble of $700$ networks per $n$ and $d$. We

317: observed that (i) the mean fraction of networks with periodic and

318: quasiperiodic windows decreases like $\sim d^{-1.3}$, (ii) the mean number

319: of windows decreases like $\sim d^{-2}$, and (iii) the window lengths

320: increase linearly with increasing $d$. These observations are insensitive

321: to increments in $s$ as long as $\Delta s \leq 0.005$. As the dimension

322: increased above $64$ the only networks with periodic windows had windows

323: that persisted for most of the $s$-interval under consideration. That is,

324: as dimension was increased, periodic windows became increasingly rare.

325: When they were observed, however, they were neither small nor intermittent,

326: but instead dominated the dynamics.

327: 

328: \vspace{-0.025in}

329: 

330: To explore the full parameter space systematically, one can fix $s$ and

331: vary the weights with random perturbations of a given size.  We surveyed

332: networks with parameters varied in a $(n(d+1)+1)$-ball with its center

333: fixed at $s$ noting that the results are insensitive to $s$ variation in

334: the chaotic portion of the $s$-interval; i.e., nearby $s$ values yield

335: identical results. Similarly, the results are insensitive to perturbation

336: size---weight variation on scales ranging from $10^{-10}$ to $1$ yield

337: similar results.

338: 

339: \vspace{-0.025in}

340: 

341: The results are shown in Fig. \ref{fig:probofwindows-high-d}: the probability

342: of observing periodic windows decreases as the dimension increases. Each

343: data point corresponds to the probability of finding a system with a periodic

344: orbit among a set of $700$ networks at a given $n$ and $d$ and each perturbed

345: $100$ times. The range of weight perturbations was $10^{-3}$ with $s=3$. We

346: found that the probability of periodic networks decreases as $d^{-2}$. Thus,

347: as dimension increases the systems are far less likely to display periodic

348: windows and, as a consequence, become more persistently chaotic. 

349: 

350: \begin{figure}[tbp]

351: \begin{center}

352: \epsfig{file=Figure2.eps, height=4.7cm, angle=270}

353: \caption{Log probability of periodic behavior versus log dimension for

354:   $700$ cases per $d$. Each case has all the weights perturbed on the

355:   order of $10^{-3}$; $100$ times per case.  The best-fit line is

356:   $\sim 1 /d^2$.

357:   }

358: \label{fig:probofwindows-high-d}

359: \end{center}

360: \end{figure}

361: %location:

362: %Users/albers/research/projects/structuralstability/data/coyote/windows_klce_stability/windowsd-d/compiled_windows_data/logdata

363: 

364: \vspace{-0.025in}

365: 

366: While this is strong evidence for the disappearance of periodic windows

367: in parameter space with increasing dimension, a stronger argument follows

368: from our observation that the fraction of networks with windows decreases

369: less quickly ($\sim d^{-1.3}$) than the overall probability of windows

370: ($\sim d^{-2}$). Thus,

371: periodic windows which do exist are concentrated in an ever-decreasing

372: fraction of networks and those with one periodic window

373: are more likely to have many periodic windows. 

374: 

375: \vspace{-0.025in}

376: 

377: To quantify the degree of $p$-chaos for an ensemble of mappings with our

378: construction consider the distribution of LCE's at a fixed $s$, noting

379: that $p$ depends on $s$.  If one takes $p$ to be the mean number of

380: positive LCE's, then $p$ increases monotonically like $\sim d/4$

381: \cite{mythesis} when $s$ is set to give the maximum number of positive

382: LCEs. If one considers $p$ to be the mean number of positive LCE's

383: minus $3$ standard deviations, a more conservative estimate, $p>0$

384: at $d=32$ and increases like $\sim 4 \log d$ at $s=3$ \cite{mythesis}.

385: We will refrain from arguing for a best definition of $p$ and simply

386: note that $p$ increases with $d$ monotonically. Together

387: with the periodic window data, this implies that while the parameter change

388: required to alter the absolute number of positive exponents decreases,

389: the perturbation required for \emph{all} the positive exponents to

390: vanish grows substantially.  Thus, the chance of falling into a periodic

391: window vanishes in proportion, and chaos becomes persistent over a

392: considerable portion of parameter space.   

393: 

394: \vspace{-0.025in}

395: 

396: These observations and detailed analysis of $400$ four-dimensional

397: dynamical systems and $200$ $64$-dimensional dynamical systems, as

398: well as many of intermediate dimension, leads to the following view of

399: dynamic (topological) variation with parameter change. All of the

400: LCEs that become positive are negative for very small and very large

401: values of $s$---the LCE spectra exhibit a single maximum. As the

402: dimension $d$ is

403: increased, their variations decrease and their $s$-dependence becomes

404: smoother; recall Fig. \ref{fig:biglced64}. Moreover,

405: with increasing dimension the number of positive exponents increases

406: monotonically \cite{mythesis}. Finally, the distance between LCE

407: zero-crossings, above the maximum, decreases with dimension

408: as shown schematically in Fig. \ref{fig:U_i}.

409: 

410: %This is all to say that, in the limit of very high

411: %dimension there are as many positive exponents as desired and they

412: %behave smoothly with parameter change. While the parameter change

413: %required to alter the absolute number of positive exponents decreases,

414: %the perturbation required for \emph{all} the positive exponents to

415: %vanish grows substantially. Thus, the chance of falling into a periodic

416: %window vanishes in proportion and chaos becomes persistent. For

417: %instance, if one considers $s$ values in the set $U = [0.2,5]$,

418: %the variation in the number of positive exponents runs from

419: %$16$ to $3$. Nevertheless, chaotic dynamics are persistent over a

420: %considerably larger portion of parameter space. The $64$-dimensional

421: %dynamical system of Fig. \ref{fig:biglced64} exhibits persistent chaos of

422: %degree $3$ ($3$-chaos) over at least the subset $[0.1, 10]$ \cite{hypviolation}.  

423: 

424: \vspace{-0.025in}

425: 

426: Figure \ref{fig:U_i} is a graphical depiction of the hypothesized

427: persistence mechanism---a plot of

428: the $s$ axis transversally intersected by LCEs. In sufficiently high

429: dimensions, the subsets $U_i$ shrink and eventually

430: fall below any resolvability. The result, then, is twofold: one observes

431: continuous topological change (bifurcations), but this is never catastrophic.

432: One sees persistent chaos of varying degrees. The onset of sufficiently

433: high dimension for this to occur for our dynamical systems was observed

434: to be $d \geq 30$. These investigations lead to the following conjecture

435: for persistent chaos in high dimensions:

436: \vspace{-0.1in}

437: \begin{conjecture}

438: \label{conjecture:klceconjecture}

439: Assume $f$ as in Eq. (\ref{equation:net1}) and a sufficiently large number $d$

440: of dimensions and number $k=n(d+2)+1$ of parameters. There exists a large

441: \footnote{``Large'' depends on $d$: for $d=64$ we estimate a Lebesgue measure

442: $m(I_s)<20$; higher $d$ remains an open question.} Lebesgue

443: measurable set of $s \in R^1$ with respect to $m_{\beta}$, $m_{w}(s)$, and

444: $m_I$, for which chaos will be degree-$p$ persistent. Moreover,

445: $p \rightarrow \infty$ as $d \rightarrow \infty$.  

446: \end{conjecture}

447: \vspace{-0.1in}

448: Since our networks are universal function approximators, this behavior

449: should be observed in typical nonlinear high-dimensional dynamical systems. 

450: 

451: \begin{figure}[tbp]

452: %\epsfig{file=U_i.eps, height=3.5cm}

453: \epsfig{file=Figure3.eps, height=2.0cm}

454: \caption{Lyapunov spectrum versus network nonlinearity $s$: $U_i$'s are the

455:   open sets in parameter space where structural stability is believed to

456:   persist. The $|U_i|$ parameter intervals shrink like $\sim d^{-1.92}$ as

457:   the dimension increases.

458:   }

459: \label{fig:U_i}

460: \end{figure}

461: 

462: \vspace{-0.025in}

463: 

464: Two comments are in order. First, the existence of chaos as a persistent

465: behavior type depends on dimension. The subset of parameter space in which

466: chaos becomes persistent increases in size (with respect to Lebesgue

467: measure) as the dimension of the dynamical system increases. This is due

468: both to the increase in the number of positive LCEs (given a sufficient

469: increase in $n$) and to a decrease in the appearance of periodic windows.

470: Second, persistence is related to the number of (linearly independent)

471: parameters in the dynamical system. The number of neurons in the network

472: effectively controls the entropy rate \cite{manffra_thesis}---that is,

473: increasing the number of neurons increases the entropy rate, number of

474: positive exponents, and the maximum of the largest exponent. Increasing

475: $n$ simply increases the degree ($p$) of the persistent chaos, but the mechanism

476: for persistent chaos remains, due to the decreasing probability of periodic

477: windows. Networks with few parameters exhibit considerably less persistent

478: chaos.

479: 

480: \vspace{-0.025in}

481: 

482: In this way high entropy-rate systems are more persistent with respect to

483: functional \emph{and} parameter perturbations. This is in accord with a wide

484: range of experimental observations of such systems. Indeed, dynamical

485: persistence is not a novel experience; often hydrodynamic engineers and plasma

486: experimentalists expend much effort in attempts to eliminate persistent

487: chaos. Here we described a mechanism in which the dynamical persistence of

488: high-dimensional systems is retained under parameter perturbation,

489: despite the fact that stricter notions of dynamical equivalence are violated.

490: This sets the stage for more specific investigations of the statistical

491: topology of stable and unstable manifolds in high-dimensional

492: systems---investigations that, one hopes, will lead to predictive scaling

493: theories for observed macroscopic properties that are grounded in

494: microscopic dynamics.

495: 

496: \vspace{-0.03in}

497: 

498: We thank J. R. Albers, R. A. Bayliss, K. Burns, W. D. Dechert, D. Feldman,

499: J. Robbin, C. R. Shalizi, and J. Supanich for helpful discussions.

500: This work was partially supported at the Santa Fe Institute under

501: the Networks Dynamics Program funded by the Intel Corporation and under the

502: Computation, Dynamics, and Inference Program via SFI's core grants from the

503: National Science and MacArthur Foundations. Direct support for DJA was

504: provided by NSF grants DMR-9820816 and PHY-9910217 and DARPA

505: Agreement F30602-00-2-0583. 

506: 

507: \vspace{-0.3in}

508: 

509: \bibliography{dstexts,partialhyperbolicity,lyapunovexponents,neuralnetworks,nilpotency,topology,analysis,structuralstability,computation,me,physics,bifurcationtheory,probability,lorenz,srb,unimodal}

510: 

511: \end{document}

512: