1: \documentclass[superscriptaddress]{revtex4}
2:
3: \usepackage{epsf}
4: \usepackage{amssymb}
5: \newcommand{\be}{\begin{equation}}
6: \newcommand{\ee}{\end{equation}}
7: \newcommand{\bea}{\begin{eqnarray}}
8: \newcommand{\eea}{\end{eqnarray}}
9: \renewcommand{\t}{\tau}
10: \newcommand{\sgn}{\mbox{sign}}
11: \newcommand{\tri}{\mbox{tri}}
12: \newcommand{\im}{i}
13:
14: \begin{document}
15:
16: \title{Information Flow through a Chaotic Channel: Prediction and
17: Postdiction at Finite Resolution}
18: \author{Richard Metzler}
19: \affiliation{New England Complex Systems Institute,
20: 24 Mt. Auburn St., Cambridge, MA 02138, USA}
21: \affiliation{Department of Physics, Massachusetts Institute of Technology,
22: Cambridge, MA 02139, USA}
23: \author{Yaneer Bar-Yam}
24: \affiliation{New England Complex Systems Institute,
25: 24 Mt. Auburn St., Cambridge, MA 02138, USA}
26: \affiliation{Department of Molecular and Cellular Biology,
27: Harvard University, Cambridge, MA 02138, USA}
28: \author{Mehran Kardar}
29: \affiliation{Department of Physics, Massachusetts Institute of Technology,
30: Cambridge, MA 02139, USA}
31:
32:
33: \begin{abstract}
34: We reconsider the persistence of information under the dynamics of
35: the logistic map in order to discuss communication through a nonlinear
36: channel where the sender can set the initial state of the system with
37: finite resolution, and the recipient measures it with the same accuracy.
38: We separate out the
39: contributions of global phase space shrinkage and local phase space
40: contraction and expansion to the uncertainty in predicting and postdicting
41: the state of the system. Thus, we determine how the
42: amplification parameter, the time lag, and
43: the resolution influence the possibility for communication.
44: A novel representation for real numbers is introduced
45: that allows for a visualization of the flow of information between
46: scales.
47: \end{abstract}
48:
49: \maketitle
50: \section{Introduction}
51: %
52: %
53: When observing a dynamical system in a given state,
54: one can ask several basic questions, two of which are, ``Where did it come
55: from?'' and ``Where is it going?'' These questions are especially
56: relevant when one wants to communicate a message by setting the initial
57: state of a system, whose state at a later time is then detected by the
58: recipient of the message. If the final state is completely uncertain, the
59: message is lost -- this is often the case in chaotic systems. On the other
60: hand, if all initial states converge to one final state, the recipient
61: cannot determine what message he was to receive -- this can occur in
62: dissipative systems. Many nonlinear systems have elements of both chaos and
63: dissipation; as an example, we choose the logistic map $f(x) = a x (1-x)$
64: \cite{Ott:Chaos}. We determine the relevance of state space shrinkage
65: and expansion for all values of the amplification parameter $a$, and
66: discuss to what extent communication in the sense mentioned is possible.
67: Note that using a chaotic channel is not something
68: the sender {\em chooses} to do -- he may
69: be forced to entrust the message to an unreliable medium. This
70: distinguishes our perspective from
71: other publications on communication through chaos such as in Refs.
72: \cite{Hayes:Communicating,Pecora:Synchronization,Bollt:Coding,
73: Rulkov:Chaotic, Baptista:Cryptography},
74: where chaotic dynamics are used in order to amplify and
75: transmit small signals.
76:
77: We will show that for very short time intervals between the initialization
78: and the measurement, the chaotic regime near $a= 4$
79: allows for optimal communication; for intermediate times, the bifurcation
80: points offer the best chances of deciphering the message; and for long
81: times, no information remains except in the bifurcation regime, where one
82: can distinguish between the branches of the cycle. Under all circumstances,
83: uncertainty about the time at which the
84: system was initialized leads to additional losses of information.
85:
86: The degree to which an observation at one time determines the result of an
87: observation at a different time is given by the mutual information between
88: these observations, and the conditional information between them.
89: The relevance of measuring information to characterize the
90: behavior of chaotic systems was realized more that 20 years ago
91: \cite{Shaw:Strange,Farmer:Information,Pompe:State,Pompe:Transinformation, Kruscha:Information}
92: and discussed in various contexts
93: \cite{Schittenkopf:Exploring,Deco:Information,Wiegerinck:Information};
94: some of the results
95: presented here have been either alluded to or derived before, especially in
96: Ref. \cite{Pompe:State}. However, for concrete calculations, most of these
97: references assumed that the system had already reached a stationary state
98: or attractor, whereas relaxation to the attractor plays a crucial role in
99: our analysis. Also, previous studies have focused on the case of
100: fully developed chaos at $a=4$; we discuss the behavior for all regimes of
101: $a$.
102:
103:
104: When inputs and outputs are measured with finite precision, it is useful to
105: have a representation of numbers that separates contributions on different
106: scales. We introduce a representation that may be superior to the usual
107: decimal or binary representations in this regard,
108: and use it to visualize the flow of
109: information between scales.
110:
111: Section \ref{SEC-info} provides an overview of
112: information-theoretical concepts, and a derivation of equations needed
113: to calculate the relevant quantities for generic maps. In
114: Sec. \ref{SEC-logmap}, we apply the formalism to the logistic map.
115: The possibility of transmitting messages by initializing
116: the system is discussed in Sec. \ref{SEC-commun}. In Sec. \ref{SEC-rep},
117: we introduce the {\em clockwork representation} of real numbers and use it to
118: illustrate the dynamics of the logistic map.
119: Section \ref{SEC-summary} summarizes the results.
120:
121:
122:
123: \section{Information-theory perspective}
124: \label{SEC-info}
125: Since observations on physical systems can only be made with finite precision,
126: the outcome can be described with a finite number of digits. Each possible
127: distinct outcome (or elementary event) can be assigned a symbol
128: that appears with a given probability, and
129: Shannon's definition of information \cite{Shannon:Mathematical}
130: can be applied to these symbols,
131: resulting in a resolution-dependent, but finite, information.
132: We divide the
133: space of possible inputs and outputs $x\in [0,1]$
134: into $r$ non-overlapping bins $i$ of uniform width $1/r$, and assume
135: that measurement precision is independent of
136: the value \cite{endnote18}.
137: %\footnote{This can be seen as defining what the observed quantity
138: % is. Transformations commonly used to switch from one set of variables/
139: % equations to another do not conserve this property.}
140: Accordingly, we
141: denote by $x_i$ the event ``a trajectory starts in bin $i$,'' and with
142: $y_j$ the event ``a trajectory ends in bin $j$.''
143: If $x_i$ is drawn from a probability distribution $P_x(x)$, the
144: information associated with $x$ is
145: \be
146: I(x) = -\sum_i P_x(x_i) \log_2 P_x(x_i);
147: \ee
148: the information $I(y)$ of $y$ can be calculated analogously from the
149: distribution $P_y(y)$.
150: The conditional information, which is needed to specify the
151: outcome $y$ given the input $x$, is
152: \be
153: I(y|x) = - \sum_i P_x(x_i) \sum_j P_{y|x}(y_j|x_i)
154: \log_2 P_{y|x}(y_j|x_j),
155: \label{LOG-condinf1}
156: \ee
157: where $P_{y|x} (y_j|x_i)$ denotes the conditional probability of $y_j$
158: occurring given that $x_i$ occurred.
159: All other quantities (mutual information $I(x\wedge y)$,
160: joint information $I(x,y)$, and conditional information $I(x|y)$
161: ) can be calculated from $I(x)$, $I(y)$, and $I(y|x)$,
162: using the set-theoretical
163: relations implied by
164: Fig. \ref{LOG-infsketch}. In particular, we make use of $I(y)=I(y\wedge
165: x)+I(y|x)$: to specify $y$, we need the mutual information $I(y\wedge x)$,
166: which represents information about $y$ that can be inferred
167: from knowledge of $x$, and the conditional information $I(y|x)$.
168:
169: \begin{figure}
170: \epsfxsize= 0.65 \columnwidth
171: \epsffile{pics/infsketch.eps}
172: \caption{For two correlated events $x$ and $y$, the relations between
173: the information $I(x)$, conditional information $I(x|y)$, joint
174: information $I(x,y)$, and mutual information $I(x\wedge y)$ are
175: illustrated in this diagram.}
176: \label{LOG-infsketch}
177: \end{figure}
178: A {\em bijective} mapping between input and output ($x$ completely determines $y$
179: and vice versa) is achieved if $I(x\wedge y)= I(y) = I(x)$.
180: Note that $I(x\wedge y)\leq \max(I_x,I_y)$ -- if the space of possible states
181: shrinks when applying $y=f(x)$, there can be at most an {\em injective}
182: mapping ($x$ completely determines $y$, but not vice versa).
183:
184:
185: We first discuss $I(y)$, then the conditional information
186: $I(y|x)$, and consider what processes influence them. We then
187: develop a formalism to derive analytic results for
188: functions that are smooth on scales comparable to the resolution,
189: e.g., if $y$ is the result of
190: applying the logistic map $f(x)=ax(1-x)$ iteratively for a small number of
191: times. The results for long times are discussed later
192: using a different approach.
193:
194: \subsection{Total information: Global phase space considerations}
195: \label{SEC-phase}
196: Under any mapping, an ensemble of input values
197: drawn from a given probability distribution
198: is generally mapped onto an ensemble of outputs that is described by another
199: distribution. It is necessary to check whether one or the other requires
200: more information to describe an associated event (a distinct value of input or
201: output).
202:
203: Sampling a continuous random variable many times with resolution $1/r$ is
204: equivalent to generating a histogram with bins of width $r$.
205: It is useful to separate the information needed to select an element
206: from this histogram
207: into two contributions: one from treating the
208: function underlying the probability distribution as continuous in $x$,
209: and another from the act of separating the input space into $r$
210: bins. Let us say that we have a probability density
211: $p_x(x)$ living on $x\in [0,1]$. The information of this distribution,
212: according to the usual
213: definition \cite{Shannon:Mathematical}, is
214: \be
215: I_c = -\int dx\; p_x(x)\; \log_2 p_x(x). \label{LOG-infodef1}
216: \ee
217: The discrete probability distribution of the histogram is
218: $P_{x} (x_i) \approx p_x(i/r)/r$. The information
219: of events drawn from this discrete distribution is
220: \be
221: I_d = -\sum_i P_x(x_i) \log_2 P_x(x_i)
222: \approx -\int dx\; r\;( p_x(x)/r) \log_2(p_x/r)= I_c + \log_2 r.
223: \label{LOG-infodef2}
224: \ee
225: Replacing the sum by the integral is valid as long as $p_x(x)$ is reasonably
226: smooth over the range of one bin (which is only roughly valid for the
227: distributions that we discuss in Sec. \ref{SEC-short}).
228:
229: If the input is drawn from a known probability distribution $p_x(x)$,
230: the output $y$ follows a probability distribution $p_y(y)$ that can be
231: calculated using the rules for transforming probability
232: distributions \cite{VanKampen},
233: \be
234: |p_x(x) dx| = |p_y(y(x))dy|, \Rightarrow p_y(y) = \sum_{\alpha} p(x_{\alpha}(y))
235: \left |\frac{dx}{dy} \right |,
236: \label{LOG-probmap}
237: \ee
238: the sum being over all $x$ which map onto $y$. Under all one-dimensional
239: chaotic maps, including the logistic map, two or more input values are
240: mapped onto the same output, a property known as {\em folding}.
241:
242:
243: \subsection{Conditional information: Local expansion and shrinkage
244: of phase space}
245: \label{SEC-expand}
246: We now study the conditional information $I(y|x)$.
247: While the last section dealt with global properties of the map,
248: here we are averaging over a local property --
249: given some $x_i$, we ask, ``how much can we know about the
250: output?'', which is independent of the behavior of the map for other input
251: values $x_{k}$.
252:
253: Let us denote the local conditional information as
254: \be
255: I_l(y|x_i) =\sum_{j} P_{y|x}(y_j | x_i) \log_2 P_{y|x} (y_j |x_i).
256: \label{LOG-Ilocal}
257: \ee
258: As long as bins are small compared to changes in $dy/dx$,
259: trajectories starting from points in the interval
260: $[x_i, x_i + 1/r]$
261: are uniformly distributed over an interval $[y_x, y_x+d/r]$, where
262: $d = |dy/dx|$. The uncertainty about the outcome is determined by the
263: overlap of this interval with the bins (as sketched in Fig.
264: (\ref{LOG-mapping})).
265: To account for this, we average
266: over the offset $o\in [0,1]$ which specifies where $y_x$ is within a bin.
267: \begin{figure}[h]
268: \epsfxsize= 0.65 \columnwidth
269: \epsffile{pics/mapping.eps}
270: \caption{Assuming small bins, trajectories from within one bin in $x$ are
271: mapped uniformly onto one or more bins in $y$.}
272: \label{LOG-mapping}
273: \end{figure}
274:
275: If a percentage $o$ of a bin is covered with trajectories, the contribution
276: of that bin to the sum in Eq. (\ref{LOG-Ilocal}) is given by
277: $I_p(o) = -(o/d) \log_2 (o/d)$. Let us look first
278: at the case $0<d<1$. In that case, either the covered interval is entirely
279: within one bin (if $o>d$) -- then the conditional information is $0$ -- or
280: the trajectories are split between two bins, resulting in a non-vanishing
281: conditional information. Averaging over $o$ gives
282: \be
283: \langle I_l(y|x)\rangle_o = \int_0^d\left (I_p(o)+I_p(d-o)\right)\, do + \int_d^1 0\; do
284: =\frac{d}{2 \ln 2}. \label{LOG-stretch1}
285: \ee
286: If $1\leq d < 2$, trajectories can be spread out over two or three bins,
287: depending on the offset:
288: \be
289: I(y|x) = \left\{ \begin{array}{ll}
290: I_p(o) + I_p(d-o) &\mbox{~~~for~~~}o>d-1 \\
291: I_p(o) + I_p(1) + I_p(d-1-o) &\mbox{~~~for~~~}o\leq d-1
292: \end{array} \right. .
293: \ee
294: Averaging yields
295: \be
296: \langle I_l(y|x)\rangle_o = \log_2 d + \frac{1}{2 d \ln 2}. \label{LOG-stretch2}
297: \ee
298: Equation (\ref{LOG-stretch2}) thus has two contributions:
299: a term logarithmic in $d$
300: to account for the bins that are fully covered, and a term from the two
301: partially covered bins, whose impact decays as $1/d$. One can show that
302: Eq. (\ref{LOG-stretch2}) is valid for any value of $d>1$.
303:
304: To find the average conditional information $I(y|x)$, we can now
305: sum over $x$, with $d(x)=|dy/dx|$:
306: \be
307: I(y|x) \approx \int \langle I_l(y|x) \rangle_o dx.
308: \label{LOG-confinfint}
309: \ee
310:
311: %Note that the derivation assumed that the offset $o$ was randomly
312: %distributed. It can be expected that this works well for large numbers of
313: %input bins with functions with a non-vanishing curvature $d^2y/dx^2$
314: %such as the logistic map), but gives quantitatively wrong results for
315: %piecewise linear maps such as $y = 2 x \bmod 1$.
316:
317: \subsection{Folding}
318: It is well known that chaotic iterative maps require a folding mechanism to
319: compensate for stretching of phase space. E.g., in the case of the logistic
320: map, the two branches of the parabola map two input points onto the same
321: output. Clearly, through this process, information about the original
322: state is lost.
323: In the framework presented so far, this is not accounted for
324: explicitly; however, it is contained implicitly in the conditional
325: information. For example, when comparing the identity function $f(x)=x$ with
326: the shift map $f(x)= 2x \bmod 1$ (which is chaotic and has folding),
327: both map the unit interval
328: uniformly onto itself and thus have the same global information $I(y)$.
329: However, the latter has a larger average local slope and thus, according to
330: Eq. (\ref{LOG-stretch2}), a larger conditional information, leading to a
331: smaller mutual information between input and output. In this case, the
332: uncertainty in prediction generated through stretching is the
333: same as that in postdiction through folding: going forward in time, one
334: does not know in which of two adjacent bins the output will be, whereas
335: looking back, one has two possible input bins that are well separated,
336: one with $x<1/2$ and one with $x>1/2$.
337:
338:
339: \section{Application to the Logistic Map}
340: \label{SEC-logmap}
341: \subsection{Basics of the Logistic Map}
342: We briefly review
343: fundamental properties of the logistic map $f(x)=a x (1-x)$ when
344: used as an iteration (i.e. $x_{t+1}= f(x_t)$). For $a\leq 1$, there is one
345: (stable) fixed point, namely $x_t = 0$. Between $1<a<3$, the only stable
346: fixed point is $x_t = 1 - 1/a$. At $a=3$, this becomes unstable and gives way
347: to a stable 2-cycle. What follows is a succession of bifurcations
348: ($n$-cycles split into $2n$-cycles) at $a\approx 3.4493$, $3.54396$,
349: $3.56438$ etc., until the cycles merge into a continuous chaotic attractor
350: at $a_c \approx 3.569946$ (see also the bifurcation diagram at the top of
351: Fig. \ref{FIG-convfour}). The chaotic regime is interrupted by smaller and
352: larger windows of periodic behavior. At $a=4$ one gets what is
353: often referred to as ``fully developed chaos;'' there,
354: the chaotic attractor spans the interval $[0,1]$.
355:
356: The Lyapunov exponent, which determines whether two trajectories starting
357: from nearby initial conditions converge or separate exponentially
358: \cite{Eckmann:Ergodic}, is
359: negative in the fixed point and bifurcation regime, becomes zero at the
360: bifurcation points, and is positive in the chaotic regime.
361:
362: We now apply the formalism developed in Secs. \ref{SEC-phase} and
363: \ref{SEC-expand} to the first few iterates of the logistic map
364: $f(x)= a x(1-x)$, then explain the long-time behavior, and give numerical
365: results on intermediate times.
366:
367:
368: \subsection{Short time behavior}
369: \label{SEC-short}
370:
371: Let us consider the probability distribution $p_y(y)$ of the first
372: iterate, starting from a uniform distribution. There are two
373: symmetric branches of $y(x)$, and Eq. (\ref{LOG-probmap}) gives
374: \be
375: p_y(y) = \left \{\begin{array}{cl}
376: \frac{2}{\sqrt{a}}\frac{1}{\sqrt{a-4y}} &\mbox{~~~~for~~~}
377: y<\frac{a}{4} \\
378: 0 & \mbox{~~~~for~~~}y\geq \frac{a}{4}
379: \end{array}\right. .
380: \ee
381: The information of this distribution can be evaluated using Eqs.
382: (\ref{LOG-infodef1},\ref{LOG-infodef2}):
383: \bea
384: I_c(y) &=& - 1 - 1/\ln 2 + \log_2 a < 0, \\ \nonumber
385: I(y)&=& -1 -1/\ln 2 + \log_2 a + \log_2 r. \label{LOG-yinfo}
386: \eea
387: In contrast, the information of the uniform
388: distribution was $I_c(x)=0$ and $I(x)=\log_2(r)$, which means that information
389: about the state of the system was lost -- $1/\ln 2 -1 \approx
390: 0.44$ bits for $a=4$, and more for $a<4$.
391:
392: The conditional information can be derived from integrating
393: Eqs. (\ref{LOG-stretch1}) and (\ref{LOG-stretch2}) over the input space.
394: In the first iterate of the logistic map, the
395: derivative is smaller than $1$ for $x\in [(1-1/a)/2, (1+1/a)/2]$, and
396: larger in the rest of the domain. Choosing boundaries appropriately and
397: making use of the symmetry of the system, we obtain
398: \be
399: \int_0^1 P(x) I_l(y|x) dx = \frac{a}{4\ln 2}
400: \ee
401: for $a<1$, and
402: \bea
403: \int_0^1 P(x) I_l(y|x) dx&=& \frac{1}{a \ln 2}\left[ \int_1^a \left(\ln x
404: +\frac{1}{2x}\right )dx + \int_0^1 \frac{x}{2} dx\right] \nonumber\\
405: &=& \frac{1}{a \ln 2} \left [(a+ 1/2) \ln a + \frac{5}{4} -a \right]
406: \label{LOG-condinfo}
407: \eea
408: for $a\geq 1$. This agrees well with numerical results for finite
409: resolution, as seen in Fig.
410: \ref{FIG-infoT1}. It should be pointed out that $I(y|x)$ does not
411: explicitly depend on the resolution, in contrast to $I(y)$.
412:
413: In Fig. \ref{FIG-infoT1}, $I(y|x)$ represents the uncertainty generated by
414: stretching and compressing; $I(x)-I(y)$ represents the uncertainty through
415: shrinking of phase space; and $I(x)-I(y\wedge x)$ is the average
416: information necessary to reconstruct $x$ from knowledge of $y$.
417: The mutual information $I(y\wedge x)$ is the amount of information about
418: the initial state retained after the mapping.
419:
420: The mutual information can be calculated by numerical integration over
421: Eq. (\ref{LOG-stretch2}) for the second and third iteration of the logistic
422: map, and good agreement with simulations is found, as shown in Fig.
423: \ref{FIG-infoT23}. For higher iterates,
424: numerical integration becomes difficult.
425: Numerical integration over the probability distribution of outputs also
426: becomes less accurate, and the approximation made in
427: Eq. (\ref{LOG-infodef2}) becomes visibly wrong for resolutions as coarse
428: as $r=100$.
429:
430: While Figs. \ref{FIG-infoT1} and \ref{FIG-infoT23} do not show a clear
431: distinction between the fixed point/cyclic regime and the chaotic regime,
432: one can see that conditional information (i.e., uncertainty
433: generated by the dynamics) increases with $a$, whereas $I(y)$ develops
434: dips. For example, the dip at $a\approx2$ represents rapid convergence to the
435: fixed point far from the bifurcation points $a=1$ and $a=3$.
436: Correspondingly, $I(y\wedge x)$ is no longer monotonic in $a$ --
437: several maxima of conserved information emerge.
438:
439: \begin{figure}
440: \epsfxsize= 0.65 \columnwidth
441: \epsffile{pics/infoT1.eps}
442: \caption{Conditional, mutual and total information
443: of input and output for $r=100$ for one
444: time step, compared to Eqs. (\ref{LOG-yinfo}) and (\ref{LOG-condinfo}).}
445: \label{FIG-infoT1}
446: \end{figure}
447:
448: \begin{figure}
449: \begin{minipage}{0.49 \columnwidth}
450: \epsfxsize= 0.98 \columnwidth
451: \epsffile{pics/infoT2.eps}
452: \end{minipage}
453: \begin{minipage}{0.49 \columnwidth}
454: \epsfxsize= 0.98 \columnwidth
455: \epsffile{pics/infoT3.eps}
456: \end{minipage}
457: \caption{Conditional, mutual and total information
458: of input and output for $r=100$ for two and three
459: time steps, compared to results of numerical integration. $I(y|x)$
460: does not depend strongly on $r$, whereas the other quantities include an
461: additive term of $\log_2 r$.}
462: \label{FIG-infoT23}
463: \end{figure}
464:
465:
466: \subsection{Long-time behavior}
467: \label{SEC-longtime}
468: For very long times, some simple statements about the information in the
469: output can be made: if the map has a single fixed point ($0<a<3$), $I(y)=0$.
470: For a cycle of length $2^n$, $y(t)$ can be defined completely
471: by stating what branch of the cycle it is on; the information
472: is therefore $n$ bits if the resolution is fine enough to resolve each
473: branch of the cycle, and each branch has an equally large basin of
474: attraction, and smaller otherwise. In the chaotic regime,
475: there is a probability distribution filling a finite fraction of the
476: interval $[0,1]$ for most values of $a$, and cycles of various lengths in
477: certain periodic windows.
478: The continuous information $I_c$ will therefore be less than one,
479: and the information at finite resolution less than or equal to $\log_2 r$, with visible
480: dips in the periodic windows. Numerical
481: results of $I(y)$ for $r=100$ and $r=10000$ shown in Fig. \ref{FIG-yinfo}
482: demonstrate these features.
483:
484: \begin{figure}
485: \epsfxsize= 0.65 \columnwidth
486: \epsffile{pics/yinfo.eps}
487: \caption{Information $I(y)$ of the output distribution for $r=100$ and
488: $r=10000$,
489: for $t\rightarrow \infty$, compared to the input information $I(x)$.}
490: \label{FIG-yinfo}
491: \end{figure}
492:
493: The mutual information is at most as large as the total information, therefore it
494: will be $0$ as well for $a<3$. In the bifurcation regime, dynamics are
495: fairly predictable. The basins of attraction for each branch of the cycle
496: are fractals, reminiscent of Cantor sets, as shown in Fig. \ref{FIG-basins}.
497: If the bins are small enough such that most bins map exclusively to one branch of a $2^n$-cycle,
498: the mutual information is of order $n$ bits.
499:
500: In the chaotic regime, information about the original state is lost at a
501: rate approximately equal to the Lyapunov exponent
502: \cite{Shaw:Strange,Pompe:State}, which here is between 0 and 1; we
503: therefore expect mutual information to be $0$ after
504: $\mathcal{O}(\log r)$ time steps. Note that this affects prediction as well
505: as postdiction: even though $I(y)$ is not much smaller than $I(x)$ for
506: $a=4$, in the absence of mutual information, it is as impossible to tell
507: where the system came from as where is is going.
508:
509: \begin{figure}
510: \epsfxsize= 0.65 \columnwidth
511: \epsffile{pics/basins.eps}
512: \caption{The 250th iterate of the logistic map at $a=3.54$, as an
513: example of the long-time dynamics.
514: The
515: dominant values are the branches of the 4-cycle; one sees the fractal
516: structure of the basins of attraction of each branch.
517: Iterates at times $250+4n$ look indistinguishable. The dotted lines
518: indicate input bins at a resolution of $r=29$ that map uniquely onto one
519: branch of the cycle (see Sec. \ref{SEC-commun}).}
520: \label{FIG-basins}
521: \end{figure}
522:
523: \subsection{Intermediate times}
524: Figure \ref{FIG-inttime}
525: shows the mutual information for $r=1000$ at various intermediate
526: times $t$, measured by
527: scanning input space with a step width small compared to the bin width.
528: Apart from the long-time features explained in Sec. \ref{SEC-longtime}, one
529: notices several peaks. The narrow peaks (e.g. near $a=2.5$)
530: occur when the fixed point is very close to the boundary between two
531: bins, such that small deviations from the fixed point lead to ambiguities
532: in the outcome. They change position if the binning is chosen differently.
533:
534: The wider peaks at $a=1$, $3$, and $3.54$ correspond to the bifurcation
535: points. There, the Lyapunov exponent is 0; deviations from the fixed point or
536: cycle decay like a power law rather than exponentially.
537: In the chaotic regime, the mutual information quickly drops to $0$, apart
538: from peaks that correspond to periodic windows (e.g. at $a\approx3.83$).
539:
540:
541: \begin{figure}
542: \epsfxsize= 0.65 \columnwidth
543: \epsffile{pics/mutinfDetails.eps}
544: \caption{Mutual information between input and output for different
545: numbers of iterations $t$, for a resolution $r=1000$. For smaller
546: resolutions, curves look similar, but more jagged. Also, the maximum
547: of the curves has an additive term of $\log_2 r$.}
548: \label{FIG-inttime}
549: \end{figure}
550:
551: \section{Communication through a logistic map channel}
552: \label{SEC-commun}
553: We now interpret and expand
554: the results of previous sections with a view to the problem of communication
555: in the following scenario: a sender A wants to transmit a message to a
556: receiver B by setting the initial state of the logistic map with some finite
557: resolution. B receives the state after $t$ iterations of the logistic map
558: and interprets it. For what values of
559: $a$ and $t$ can A expect any degree of transmission, and what resolution do
560: A and B need?
561:
562: The scenario may seem contrived; however, setting the initial state of some
563: physical system (like a sheet of paper, or a hard disk) in the hope that
564: someone will be able to read it {\em is} the usual way of transmitting
565: messages over long times. Usually people choose systems whose dynamics are
566: somewhat stable to perturbations and slow compared to the time $t$, but they
567: may not always have that choice.
568: Let us look at two different communication problems.
569:
570: \subsection{Reconstructing the initial state}
571: In this scenario,
572: A wants B to know the state that A started the system
573: in. The problem is then essentially one of postdiction for the recipient,
574: and the relevant quantity is the difference between the mutual information
575: $I(y\wedge x)$
576: and the input information $I(x)$. Surprisingly, for very short
577: times, Figs. \ref{FIG-infoT1} and \ref{FIG-infoT23} show that
578: B has the best chances of postdicting the initial state in the
579: chaotic regime near $a=4$ -- the loss of information through chaos is
580: not as significant as that through phase space shrinking in the low-$a$-regime.
581: Note that at least one bit is lost: since $f(x)$ is symmetric around $x=1/2$, it is impossible to tell whether the system was started on the left or right branch.
582:
583: For intermediate times, at high $a$, chaotic dynamics eliminates all
584: information about the initial state; so does fast convergence to a single fixed
585: point for small $a$. As Fig. \ref{FIG-inttime} shows,
586: B's situation is best if $a$ is close to one of the
587: bifurcation points, where convergence follows a power law rather than an
588: exponential.
589:
590: In the limit of very long times, when the system has converged to its
591: attractor, the only regime where information about the initial state
592: persists is the bifurcation regime. What value of $a$ gives optimal
593: transmission depends on the resolution: the recipient has to be able to
594: resolve all branches of the cycle to make full use of the remaining
595: information.
596:
597: In all three time regimes, it is important for the recipient to know the
598: precise time at which the system was initialized. The information required
599: to specify the time lag has to be included in the conditional information
600: $I(x|y)$. To give a simple example: in the long-time regime, if the time
601: is either $t$ or $t+1$ with probability $1/2$, an additional bit of
602: information is required to reconstruct the initial state.
603:
604:
605: \subsection{Determining the final state with certainty}
606: The second communication problem is this: A only wants to send B a
607: message that B can decode with certainty: A chooses the initial state
608: (again with resolution $r$) such that all trajectories from that state end
609: in one output bin. The relevant questions are now, what resolution do A and
610: B need to specify at least two different final states for various times
611: and amplifications; and, given a certain resolution, how many distinct final
612: states exist that can be achieved with certainty by choosing an appropriate
613: initial state? This is a problem of prediction on the part of the sender.
614:
615: The answer is clearest for long times: as described above, the only regime
616: where information persists and communication is possible at all
617: is that of cyclic behavior. The sender needs
618: to identify input bins that lie completely in the basin of attraction of
619: one branch of the cycle, such that each initial value in the bin leads to
620: the same final value. One such bin should be found
621: for each distinct branch, and the resolution must be
622: sufficient for the recipient to identify each branch (see
623: Fig. \ref{FIG-basins} for an example of such a set of bins). Numerics
624: show that the latter constraint is weaker than the first: it is the
625: sender's resolution that limits communication.
626: We find that a resolution of 7 bins is sufficient for the 2-cycle at values of
627: $a= 3.5$; thus, $\log_2 7 \approx 2.8$ bits are needed to specify the
628: input to transfer one bit to the recipient. For longer cycles, the ratio
629: can be more efficient: 29 bins are enough to resolve the 4-cycle at
630: $a=3.54$, yielding $4.86$ bits of input for 2 bits of output. At $a=3.562$,
631: a resolution of 122 specifies each branch of the 8-cycle, giving 3 bits of
632: output for $6.93$ bits of input. (We assume that $a$ is constant and
633: precisely known to the sender and recipient.)
634:
635:
636: In the short and intermediate time regimes, the number of distinct final
637: states that can be reached with certainty increases roughly linearly with
638: resolution. The slope is a function of both $t$ and $a$,
639: and its value indicates the amount of information
640: lost in the channel. Fig. \ref{FIG-finalstates} shows the number
641: of distinct states at $r=1000$ for different times and amplifications.
642: Some features are similar to Fig. \ref{FIG-inttime}: for longer times,
643: peaks at the bifurcation points emerge, representing the slow loss of
644: information. Between the bifurcation points, plateaus at
645: values of 2 and 4 can be seen for $t=40$. In the chaotic regime,
646: the number of predictable final states goes to zero with increasing time.
647:
648: The most surprising feature of Fig. \ref{FIG-finalstates} is that
649: the curve for $t=1$ is not monotonic, and even drops below those for
650: longer times. The reason is that the first iterate has a slope greater than
651: $1$ for
652: most $x$ values at high $a$, which makes unique prediction impossible.
653: Further iterations can re-compress parts of the state space that were
654: stretched in the first iteration, leading to a better match between input
655: and output bins.
656:
657:
658:
659: \begin{figure}
660: \epsfxsize= 0.65 \columnwidth
661: \epsffile{pics/finalstates.eps}
662: \caption{The number of distinct final states that can be reached with
663: certainty by choosing an appropriate initial state. Both sender and
664: recipient use $r=1000$.}
665: \label{FIG-finalstates}
666: \end{figure}
667:
668:
669:
670: \section{Scale-resolving representations of real numbers}
671: \label{SEC-rep}
672: Dissipation and chaos have a common aspect: in both cases the dynamics
673: makes a connection between large and small scales. Chaotic dynamics amplify
674: small differences in the initial states until they reach macroscopic
675: proportions, whereas dissipative dynamics shrink differences until they
676: vanish below the threshold of perception.
677: To represent this adequately, we
678: first have to make clear what we mean by
679: information on different scales.
680:
681: Let us consider real numbers $x$. On one hand, each real number can be
682: represented by one point on the real axis -- it is a
683: one-dimensional quantity. On the other hand, in the usual descriptions
684: (decimal, binary etc.), real numbers are represented by a set of integers
685: that stand for different scales -- the scales of 1s, 10s, 100s, etc.
686: This makes sense because it reflects what happens when some $\Delta x$ is
687: added to $x$: the digits of $x$ are strongly affected for all scales finer
688: than the scale of $\Delta x$, and weakly
689: affected for coarser scales.
690:
691: There are two problems with the usual representations: first, they are
692: {\em discontinuous}: a small change on a fine scale will have no effect at
693: all on coarser scales most of the time, but a dramatic effect in rare cases
694: (such as when $0.001$ is added to $0.999$). This is a necessary side effect
695: of using discrete (integer) representations on each scale.
696: Second, they do not lend themselves to simple modification under
697: {\em multiplication}: multiplication is basically a convolution of the
698: representations of the two factors.
699: Whereas multiplications with numbers that have a
700: simple representation in the chosen base gives a shift
701: (e.g. multiplication with 10 in the decimal representation just shifts the
702: decimal point), all other factors lead to changes throughout the scales.
703: It may therefore be worthwhile to think about alternative representations
704: for which the information content of various scales is easy to visualize.
705:
706: First, we need to decide what properties we expect the representation to
707: have. At a very basic level, each real number should map onto one
708: representation, and each representation should map onto at most one
709: real number. (Since we want to include the option of representing one real
710: number by a set of real or complex numbers,
711: a bijective mapping is not generally
712: possible.) Also, elements of the representation
713: corresponding to finer scales should not
714: include the information at coarser scales -- otherwise they would not be
715: specific to their scale. One way of achieving this is using periodic
716: functions, with the period equal to the scale to be studied.
717:
718: \subsection{The Clockwork Representation}
719: Using the most natural periodic function, one obtains what we call
720: the {\em clockwork representation} (CR).
721: It maps each real number $x$ onto set of complex numbers
722: \be
723: c_j(x) = \exp(2\pi \im\; 2^j x),
724: \ee
725: with $j$ identifying the scale. The base $2$ is was chosen in analogy to
726: the familiar binary representation -- any number
727: other than 1 can also be used. The term ``clockwork representation'' was chosen because
728: each digit can be thought of as a cog in a clockwork of consecutively
729: smaller cogs, as depicted in Fig. \ref{clockworkfig1}
730: -- two turns of $c_2$ cause one turn of $c_1$, but four turns
731: of $c_3$. Using a different base is equivalent to using cogs of a different
732: radius ratio.
733:
734: We can use the usual laws for exponential functions to see how addition and
735: multiplication of numbers carries over to their clockwork representation:
736: \bea
737: c_j(x+y) &=& \exp(2 \pi \im\; 2^j x) \exp(2 \pi \im\; 2^j y) =
738: c_j(x)\,c_j(y);\\
739: c_j(ax) &=& \exp(2 \pi \im \; 2^{j + \log_2 a} x) = c_{j+\log_2 a} x.
740: \eea
741: The second line appears a little problematic for two reasons: first,
742: it introduces
743: an asymmetry between the object under consideration $x$ and the factor $a$
744: -- the result might as well have been written as $c_{j+\log_2 x} a$.
745: This is, in some cases, desired: there is often a conceptual difference
746: between the dynamical variables and the parameters of a model.
747: Second, scales become continuous rather than discrete. However,
748: this is more of an advantage rather than a disadvantage.
749: The outcome is well-defined even for non-integer scales,
750: as opposed to the case of discrete
751: representations: one can write a number in base 2 or base 3, but not base
752: $2.5$.
753:
754: The CR gives a set of complex numbers; while it is clear how to do
755: mathematical operations on them, it is not completely obvious how to
756: {\em display} them. It can be argued that usually one is interested in the
757: imaginary part: it gives $0$ for scales much coarser than that of the number
758: under consideration, and it gives $0$ for fine scales if $x$ is a power of
759: $2$, much like the bits in a binary representation would.
760:
761:
762: \begin{figure}
763: \epsfxsize= 0.65 \columnwidth
764: \epsffile{pics/clockwork1.eps}
765: \caption{The clockwork representation; each digit represents one cog.}
766: \label{clockworkfig1}
767: \end{figure}
768:
769: Another way to think about the CR is to look at the continuous
770: function that generates the digits, as in
771: Fig. \ref{clockworkfig2}: it shows $\sin(2 \pi 2^j)$, which is the
772: imaginary part of the CR of the number $1$. It has an exponential tail
773: toward coarse scales and an oscillating part with a frequency that
774: increases exponentially toward fine scales. The CR picks out the values of
775: this function at integer values of $j$. For any other number $x$, shift the
776: curve to the left by $\log_2 x$, and again pick the values at integer
777: values of $j$.
778: \begin{figure}
779: \epsfxsize= 0.65 \columnwidth
780: \epsffile{pics/clockwork2.eps}
781: \caption{The clockwork representation of the number 1:
782: each digit corresponds to the
783: value of this function at integer values of $j$.}
784: \label{clockworkfig2}
785: \end{figure}
786:
787:
788: \subsection{Using the CR in the logistic map}
789: As explained in the previous sections, the clockwork representation can
790: give an impression about how numbers change on different scales of
791: resolution. This can be applied to illustrate the behavior of the logistic
792: map. The most obvious case is that of convergence to a fixed point or
793: cycle, as shown in Fig. \ref{FIG-conv}: for $a=2.95$ (slightly below the
794: first period doubling), a random initial state (left side) converges to
795: the fixed point (right side); differences from the final state occur on
796: finer and finer scales as time progresses, resulting in a ridge traveling
797: to finer scales.
798: \begin{figure}
799: \epsfxsize= 0.65 \columnwidth
800: \epsffile{pics/log_convergence.eps}
801: \caption{Imaginary part of the CR of the $t$th iterate of the
802: logistic map, at $a=2.95$. Deviations from the fixed point travel
803: ``downstream,'' to finer scales.}
804: \label{FIG-conv}
805: \end{figure}
806: This ridge also shows up in a two-dimensional Fourier transform of the CR,
807: seen in Figs. \ref{FIG-convfour}:
808: for $a=2.8$, the absolute squared Fourier
809: transform of the CR shows diagonal ridges. To make the effect clearer, the
810: plot averaged over 100 uniformly distributed initial conditions. For
811: $a=3.05$, one sees a horizontal stripe in the center of the figure
812: in addition to the diagonal
813: structures, indicating that the final state has a period of 2.
814:
815: The slope of the diagonal structures during convergence to a limit cycle
816: is directly proportional to the Lyapunov exponent $\lambda$: during each time
817: step, the deviation from the limit cycle diminishes by a factor of
818: $\exp{\lambda}$. In a plot of $\log_2 x$ vs. $t$ such as Fig.\ref{FIG-conv},
819: the ridge therefore has a slope of $\lambda/\ln 2$. In a Fourier transform
820: plotted as frequency vs. inverse scale, the ridge causes diagonal
821: structures of the same slope. To illustrate this, dashed lines of slope
822: $\lambda/\ln 2$ are shown in Fig. \ref{FIG-convfour}.
823:
824:
825: \begin{figure}
826: \epsfxsize= 0.8 \columnwidth
827: \epsffile{pics/convergence.eps}
828: \caption{Squared amplitude of the Fourier transform of the CR, averaged
829: over more than $100$ initial conditions, at $a=2.8$, $3.05$, $3.7$, and
830: $4.0$. Dashed lines indicate a slope $\lambda/\ln 2$, where $\lambda$
831: is the Lyapunov exponent.}
832: \label{FIG-convfour}
833: \end{figure}
834: It is not obvious that this holds for the chaotic regime as well: there,
835: $x$ keeps changing on all scales, not just increasingly small ones, and
836: multiplication induces a folding of small and large scales.
837: Interestingly, however, similar structures can be found in the chaotic regime as
838: well: even though a 3D plot of the CR versus time looks unstructured and
839: chaotic, a Fourier transform averaged over sufficiently many
840: initial conditions often reveals diagonal stripes (Fig. \ref{FIG-convfour},
841: bottom), this time tilted in the opposite way
842: -- which indicates information traveling to coarser scales rather
843: than finer ones. Although the structures are less clear than for convergence,
844: an approximate correspondence between the slope of the structures and the Lyapunov
845: exponent still holds for many values of $a$,
846: as shown by the diagonal lines in the plots. In other cases, the structures
847: are less clear:
848: in particular, for $a=4.0$ the plot shows strong diagonal structures with a slope of
849: 2 overlaid on a weak diagonal with a slope of approximately $1$.
850: The latter is expected from the value of the Lyapunov exponent.
851:
852: It should be noted that using the binary representation instead of the
853: CR yield similar pictures (albeit more noisy) for the convergence to a
854: fixed point, but shows no discernible structures for the convergence to the
855: chaotic attractor.
856:
857: \section{Summary}
858: \label{SEC-summary}
859: We have presented a study of the loss of information through the nonlinear
860: dynamics of the logistic map, using analytical means for short times and
861: numerics and heuristic arguments for long times.
862: As Secs. \ref{SEC-info} and \ref{SEC-commun} have shown,
863: different processes are
864: relevant in the different regimes:
865: in the small-$a$ regime, shrinking phase space quickly makes postdiction
866: impossible and prediction trivial, thus eliminating the possibility of
867: communication.
868: In the chaotic regime (very high $a$), phase space is largely conserved, but
869: sensitivity to initial conditions prevents both
870: prediction and postdiction for longer times.
871: The bifurcation regime provides a middle
872: ground: some information about the initial state persists, determining what
873: branch of the cycle one finds the system in. The basins of attraction for
874: the branches have a fractal structure, which means that some large
875: intervals of initial values exist that lead to one branch with certainty.
876: At the bifurcation points, convergence to the final state is slow, which
877: makes some transfer of information possible for intermediate times.
878:
879: The clockwork representation introduced in Sec. \ref{SEC-rep} is a
880: continuous generalization of the usual discrete (binary, decimal etc.)
881: representations in which
882: addition and multiplication of two objects are more transparent
883: than in the discrete case, and which allows for a more elegant
884: visualization of the flow of information between scales and the convergence
885: to fixed points. It is even possible to identify the slope of structures in
886: the Fourier transform of the CR with the Lyapunov exponent of the map.
887: While the CR thus seems to be a conceptual and visual tool of some use,
888: it remains to be seen whether this representation
889: will find additional applications in analyzing dynamical systems.
890:
891: \begin{acknowledgments}
892: This work is partially supported by NSF through grant No. DMS-0083885.
893: M.K. is supported by NSF through grant No. DMR-01-18213.
894: \end{acknowledgments}
895:
896: %\bibliography{bibliography}
897:
898: \begin{thebibliography}{17}
899: \expandafter\ifx\csname natexlab\endcsname\relax\def\natexlab#1{#1}\fi
900: \expandafter\ifx\csname bibnamefont\endcsname\relax
901: \def\bibnamefont#1{#1}\fi
902: \expandafter\ifx\csname bibfnamefont\endcsname\relax
903: \def\bibfnamefont#1{#1}\fi
904: \expandafter\ifx\csname citenamefont\endcsname\relax
905: \def\citenamefont#1{#1}\fi
906: \expandafter\ifx\csname url\endcsname\relax
907: \def\url#1{\texttt{#1}}\fi
908: \expandafter\ifx\csname urlprefix\endcsname\relax\def\urlprefix{URL }\fi
909: \providecommand{\bibinfo}[2]{#2}
910: \providecommand{\eprint}[2][]{\url{#2}}
911:
912: \bibitem[{\citenamefont{Ott}(1993)}]{Ott:Chaos}
913: \bibinfo{author}{\bibfnamefont{E.}~\bibnamefont{Ott}},
914: \emph{\bibinfo{title}{Chaos in Dynamical Systems}}
915: (\bibinfo{publisher}{Cambridge University Press},
916: \bibinfo{address}{Cambrigde}, \bibinfo{year}{1993}).
917:
918: \bibitem[{\citenamefont{Hayes et~al.}(1993)\citenamefont{Hayes, Grebogi, and
919: Ott}}]{Hayes:Communicating}
920: \bibinfo{author}{\bibfnamefont{S.}~\bibnamefont{Hayes}},
921: \bibinfo{author}{\bibfnamefont{C.}~\bibnamefont{Grebogi}}, \bibnamefont{and}
922: \bibinfo{author}{\bibfnamefont{E.}~\bibnamefont{Ott}},
923: \bibinfo{journal}{Phys. Rev. Lett.} \textbf{\bibinfo{volume}{70}},
924: \bibinfo{pages}{3031} (\bibinfo{year}{1993}).
925:
926: \bibitem[{\citenamefont{Pecora and Carroll}(1990)}]{Pecora:Synchronization}
927: \bibinfo{author}{\bibfnamefont{L.~M.} \bibnamefont{Pecora}} \bibnamefont{and}
928: \bibinfo{author}{\bibfnamefont{T.~L.} \bibnamefont{Carroll}},
929: \bibinfo{journal}{Phys. Rev. Lett.} \textbf{\bibinfo{volume}{64}},
930: \bibinfo{pages}{821} (\bibinfo{year}{1990}).
931:
932: \bibitem[{\citenamefont{Bollt et~al.}(1997)\citenamefont{Bollt, Lai, and
933: Grebogi}}]{Bollt:Coding}
934: \bibinfo{author}{\bibfnamefont{E.}~\bibnamefont{Bollt}},
935: \bibinfo{author}{\bibfnamefont{Y.-C.} \bibnamefont{Lai}}, \bibnamefont{and}
936: \bibinfo{author}{\bibfnamefont{C.}~\bibnamefont{Grebogi}},
937: \bibinfo{journal}{Phys. Rev. Lett.} \textbf{\bibinfo{volume}{79}},
938: \bibinfo{pages}{3787} (\bibinfo{year}{1997}).
939:
940: \bibitem[{\citenamefont{Rulkov et~al.}(2002)\citenamefont{Rulkov, Vorontsov,
941: and Illing}}]{Rulkov:Chaotic}
942: \bibinfo{author}{\bibfnamefont{N.~F.} \bibnamefont{Rulkov}},
943: \bibinfo{author}{\bibfnamefont{M.~A.} \bibnamefont{Vorontsov}},
944: \bibnamefont{and} \bibinfo{author}{\bibfnamefont{L.}~\bibnamefont{Illing}}
945: (\bibinfo{year}{2002}), \bibinfo{note}{nlin.CD/0211019}.
946:
947: \bibitem[{\citenamefont{Baptista}(1998)}]{Baptista:Cryptography}
948: \bibinfo{author}{\bibfnamefont{M.~S.} \bibnamefont{Baptista}},
949: \bibinfo{journal}{Phys. Lett. A} \textbf{\bibinfo{volume}{1998}},
950: \bibinfo{pages}{50} (\bibinfo{year}{1998}).
951:
952: \bibitem[{\citenamefont{Shaw}(1981)}]{Shaw:Strange}
953: \bibinfo{author}{\bibfnamefont{R.}~\bibnamefont{Shaw}}, \bibinfo{journal}{Z.
954: Naturforsch.} \textbf{\bibinfo{volume}{36}}, \bibinfo{pages}{80}
955: (\bibinfo{year}{1981}).
956:
957: \bibitem[{\citenamefont{Farmer}(1982)}]{Farmer:Information}
958: \bibinfo{author}{\bibfnamefont{J.~D.} \bibnamefont{Farmer}},
959: \bibinfo{journal}{Z. Naturforsch.} \textbf{\bibinfo{volume}{37}},
960: \bibinfo{pages}{1304} (\bibinfo{year}{1982}).
961:
962: \bibitem[{\citenamefont{Pompe et~al.}(1986)\citenamefont{Pompe, Kruscha, and
963: Leven}}]{Pompe:State}
964: \bibinfo{author}{\bibfnamefont{B.}~\bibnamefont{Pompe}},
965: \bibinfo{author}{\bibfnamefont{J.}~\bibnamefont{Kruscha}}, \bibnamefont{and}
966: \bibinfo{author}{\bibfnamefont{R.~W.} \bibnamefont{Leven}},
967: \bibinfo{journal}{Z. Naturforsch.} \textbf{\bibinfo{volume}{41}},
968: \bibinfo{pages}{801} (\bibinfo{year}{1986}).
969:
970: \bibitem[{\citenamefont{Pompe and Leven}(1986)}]{Pompe:Transinformation}
971: \bibinfo{author}{\bibfnamefont{B.}~\bibnamefont{Pompe}} \bibnamefont{and}
972: \bibinfo{author}{\bibfnamefont{R.~W.} \bibnamefont{Leven}},
973: \bibinfo{journal}{Physica Scripta} \textbf{\bibinfo{volume}{34}},
974: \bibinfo{pages}{8} (\bibinfo{year}{1986}).
975:
976: \bibitem[{\citenamefont{Kruscha and Pompe}(1988)}]{Kruscha:Information}
977: \bibinfo{author}{\bibfnamefont{K.}~\bibnamefont{Kruscha}} \bibnamefont{and}
978: \bibinfo{author}{\bibfnamefont{B.}~\bibnamefont{Pompe}}, \bibinfo{journal}{Z.
979: Naturforsch.} \textbf{\bibinfo{volume}{43}}, \bibinfo{pages}{93}
980: (\bibinfo{year}{1988}).
981:
982: \bibitem[{\citenamefont{Schittenkopf and Deco}(1996)}]{Schittenkopf:Exploring}
983: \bibinfo{author}{\bibfnamefont{C.}~\bibnamefont{Schittenkopf}}
984: \bibnamefont{and} \bibinfo{author}{\bibfnamefont{G.}~\bibnamefont{Deco}},
985: \bibinfo{journal}{Physica D} \textbf{\bibinfo{volume}{94}},
986: \bibinfo{pages}{57} (\bibinfo{year}{1996}).
987:
988: \bibitem[{\citenamefont{Deco et~al.}(1997)\citenamefont{Deco, Schittenkopf, and
989: Sch{\"{u}}rmann}}]{Deco:Information}
990: \bibinfo{author}{\bibfnamefont{G.}~\bibnamefont{Deco}},
991: \bibinfo{author}{\bibfnamefont{C.}~\bibnamefont{Schittenkopf}},
992: \bibnamefont{and}
993: \bibinfo{author}{\bibfnamefont{B.}~\bibnamefont{Sch{\"{u}}rmann}},
994: \bibinfo{journal}{Int. J. Bifurc. Chaos} \textbf{\bibinfo{volume}{7}},
995: \bibinfo{pages}{97} (\bibinfo{year}{1997}).
996:
997: \bibitem[{\citenamefont{Wiegerinck and
998: Tennekes}(1990)}]{Wiegerinck:Information}
999: \bibinfo{author}{\bibfnamefont{W.}~\bibnamefont{Wiegerinck}} \bibnamefont{and}
1000: \bibinfo{author}{\bibfnamefont{H.}~\bibnamefont{Tennekes}},
1001: \bibinfo{journal}{Phys. Lett. A} \textbf{\bibinfo{volume}{144}},
1002: \bibinfo{pages}{145} (\bibinfo{year}{1990}).
1003:
1004: \bibitem[{\citenamefont{Shannon}(1948)}]{Shannon:Mathematical}
1005: \bibinfo{author}{\bibfnamefont{C.~E.} \bibnamefont{Shannon}},
1006: \bibinfo{journal}{The Bell Systen Technical Journal}
1007: \textbf{\bibinfo{volume}{27}}, \bibinfo{pages}{379}
1008: (\bibinfo{year}{1948}).
1009:
1010: \bibitem{endnote18}{This can be regarded as defining what the observed quantity is. Transformations commonly used to switch from one set of variables/equations to another do not conserve this property.}
1011:
1012:
1013: \bibitem[{\citenamefont{van Kampen}(1992)}]{VanKampen}
1014: \bibinfo{author}{\bibfnamefont{N.}~\bibnamefont{van Kampen}},
1015: \emph{\bibinfo{title}{Stochastic Processes in Physics and Chemistry}}
1016: (\bibinfo{publisher}{Elsevier Science B.V.}, \bibinfo{address}{Amsterdem},
1017: \bibinfo{year}{1992}).
1018:
1019: \bibitem[{\citenamefont{Eckmann and Ruelle}(1985)}]{Eckmann:Ergodic}
1020: \bibinfo{author}{\bibfnamefont{J.-P.} \bibnamefont{Eckmann}} \bibnamefont{and}
1021: \bibinfo{author}{\bibfnamefont{D.}~\bibnamefont{Ruelle}},
1022: \bibinfo{journal}{Rev. Mod. Phys.} \textbf{\bibinfo{volume}{57}},
1023: \bibinfo{pages}{617} (\bibinfo{year}{1985}).
1024:
1025: \end{thebibliography}
1026:
1027:
1028: \end{document}
1029:
1030: