1: \documentclass[aps,showkeys,groupedaddress,twocolumn,showpacs]{revtex4/revtex4}
2: \usepackage{graphicx}
3: \usepackage[ansinew]{inputenc}
4: \usepackage[tbtags]{amsmath}
5: \usepackage{amssymb}
6: \usepackage{epsfig}
7: \newcommand{\erfc}{\mbox{erfc}}
8: \newcommand{\erf}{\mbox{erf}}
9: \newcommand{\joint}{$\rho^{\Theta}_j(x_{n},x_{n+1},d)$ }
10: \newcommand{\ja}{$\rho^{\Theta}_j(x_{n+1},x_n,a,eta)$ }
11: \newcommand{\marg}{$\rho^{\Theta}_m(x_n,d)$ }
12: \newcommand{\ma}{$\rho^{\Theta}_m(x_n,a,\eta)$ }
13: \newcommand{\ptotal}{$\rho^{\Theta}(a,\eta)$ }
14: \newcommand{\antima}{$\bar{\rho}^{\Theta}_m(x_n,a,\eta)$ }
15: \begin{document}
16: \title{When are Extreme Events the better predictable, the larger they are?}
17: \author{S. Hallerberg, H. Kantz}
18: \affiliation{
19: Max Planck Institute for the Physics of Complex Systems\\
20: N\"othnitzer Str.\ 38, D 01187 Dresden, Germany\\
21: }
22:
23: \date{\today}
24: \begin{abstract}
25: We investigate the predictability of extreme events in time series. The focus of this work is to understand under which circumstances large events are better predictable than smaller events.
26: %
27: Therefore we use a simple prediction algorithm based on precursory structures which are identified using the maximum likelihood principle.
28: %
29: Using the receiver operator characteristic curve as a measure for the quality
30: of predictions we find that the dependence on the event magnitude is closely linked
31: to the probability distribution function of the underlying stochastic
32: process.
33: %
34: We evaluate this dependence on the probability distribution function analytically
35: and numerically.
36: %
37: If we assume that the optimal precursory structures are used to make the
38: predictions, we find that large increments are better predictable if the
39: underlying stochastic process has a Gaussian probability distribution
40: function, whereas larger increments are harder to predict if the underlying
41: probability distribution function has a power law tail.
42: %
43: In the case of an
44: exponential distribution function we find no significant dependence on the event magnitude.
45: %
46: Furthermore we compare these results with predictions of increments in
47: correlated data, namely, velocity increments of a free jet flow. The velocity
48: increments in the free jet flow are in dependence on the time scale
49: either asymptotically Gaussian or asymptotically exponential distributed.
50: The numerical
51: results for predictions within free jet data are in good agreement with the previous analytical considerations for random numbers.
52: \end{abstract}
53: \pacs{02.50.-r,05.45.Tp}
54: \keywords{time series analysis, extreme events, extreme increments,
55: statistical inference, likelihood ratio}
56: \maketitle
57: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
58: \section{Introduction}
59: Systems with a complex time evolution, which generate
60: a great impact event from time to time, are ubiquitous.
61: %
62: Examples include
63: fluctuations of prices for financial assets in economy with rare market
64: crashes, electrical activity of human brain with rare epileptic seizures,
65: seismic activity of the earth with rare earthquakes, changing weather
66: conditions with rare disastrous storms, and also fluctuations of online
67: diagnostics of technical machinery and networks with rare breakdowns or
68: blackouts.
69: %
70: Due to the complexity of the systems mentioned, a complete
71: modeling is usually impossible, either due to the huge number of degrees of freedom involved, or due to a lack of precise knowledge about the governing
72: equations.
73: %
74: This is why one applies the framework of prediction via precursory structures
75: for such cases. The typical application for prediction with precursory
76: structures is a prediction of an event which occurs in the very near future,
77: i.e., on short timescales compared to the lifetime of the system.
78: %
79: A classical example for the search for precursory structures is the prediction
80: of earthquakes \cite{Jackson}.
81: %
82: A more recently studied example is the short
83: term prediction of strong turbulent wind gusts, which can destroy wind
84: turbines \cite{Physa, Euromech}.
85:
86: In a previous work \cite{Sarah2}, we studied the quality of predictions
87: analytically via precursory structures for increments in an AR(1) process and
88: numerically in a long-range correlated ARMA process.
89: %
90: The long-range
91: correlations did not alter the general findings for Gaussian processes,
92: namely, that larger events are better predictable.
93:
94: Furthermore we found other works which report the same effect for earthquake
95: prediction \cite{Fatehme}, prediction of avalances in SOC-models
96: \cite{Sandpile} and in multiagent games \cite{Johnson1}.
97: %
98: In this contribution, we investigate the influence of the probability
99: distribution function (PDF) of the noise
100: term in detail by using not only Gaussian, but also exponential and power-law
101: distributed noise.
102: %
103: This approach is also motivated by the book of Egans \cite{Egans}
104: which explains that receiver operator characteristics (ROC) obtained in signal detection problems can be
105: ordered families of functions in dependence on a parameter.
106: %
107: We are now interested in learning how the behavior of these families of functions depends on the event magnitude and the distribution of the stochastic process,
108: if the ROC curve is used for evaluating the quality of predictions.\\
109: %
110: After defining the prediction scheme in Sec.\ \ref{pre} and the method for
111: measuring the quality of a prediction in Sec.\ \ref{roc},
112: %
113: we explain in Sec.\ \ref{evsize} how to consider the influence on the event magnitude.
114: %
115: In Sec.\ \ref{constr} we formulate a constraint, which has to be fulfilled in order to find a better predictability of larger (smaller) events.
116: %
117: In the next section, we apply this constraint to compare the quality of
118: predictions of large increments within Gaussian (Sec.\ \ref{Gaussian}), exponential distributed
119: (Sec.\ \ref{symexpo}) and power-law distributed i.i.d.\ random numbers (Sec.\ \ref{powl}).
120: %
121: We study the prediction of increments in free jet data in Sec.\ \ref{freejet}.
122: %
123: Conclusions appear in Sec.\ \ref{conclusions}.
124: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
125: \section{Definitions and setup\label{sec2}}
126: The considerations in this section are made for a time series \cite{Box-Jen,Brockwell}, i.e.\ , a set of measurements $x_n$ at discrete
127: times $t_n$, where $t_n = t_0 + n\Delta$ with a sampling
128: interval $\Delta$ and $ n \in \mathbb{N}$.
129: %
130: The recording should contain sufficiently many extreme events so that we are able to extract statistical
131: information about them.
132: %
133: We also assume that the event of
134: interest can be identified on the basis of the observations, e.g.\, by the
135: value of the observation function exceeding some threshold, by a sudden
136: increase, or by its variance exceeding some threshold.
137: %
138: We express the presence (absence) of an event by using a binary variable $Y_{n+1}$.
139: %
140: \begin{eqnarray}
141: Y_{n+1} & = &\left\{ \begin{array}{ll}
142: 1 \quad& \mbox{an event occurred at time} \\
143: & n+1 \\
144: 0 \quad& \mbox{no event occurred at time} \\
145: & n+1\end{array} \right.
146: \end{eqnarray}
147: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
148: \subsection{The choice of the precursor \label{pre}}
149: When we consider prediction via precursory structures ({\sl precursors},
150: or {\sl predictors}), we are typically in a situation where we assume that the
151: dynamics of the system under study has both, a deterministic and a stochastic
152: part.
153: %
154: The deterministic part allows one to assume that there is a relation between
155: the event and its precursory structure which we can use for predictive
156: purposes.
157: %
158: However, if the dynamic of the system was fully deterministic
159: there would be no need to predict via precursory structures, but we could
160: exploit our knowledge about the dynamical system as it is done, e.g., in weather forecasting.
161:
162: In this contribution we focus on the influence of the stochastic part of the
163: dynamics and assume therefore a very simple deterministic correlation between
164: event and precursor.%
165: The presence of this stochastic part determines that we cannot expect the precursor to preced {\em every} individual event.
166: %
167: That is why we define a precursor in this context as a data structure which is {\em typically} preceding an event, allowing deviations from the given structure, but also allowing events without
168: preceeding structure.
169:
170: For reasons of simplicity the following considerations are made for precursors
171: in real space, i.e., structures in the time series.
172: %
173: However, there is no reason not to apply the same ideas for precursory structures, which live in phase space.
174:
175: In order to predict an event $Y_{n+1}$ occurring at the time $(n+1)$ we compare the last $k$ observations, to which we will refer as the {\sl precursory variable}
176: %
177: \begin{equation}
178: {\mathbf x}_{(n-k+1,n)}= (x_{n-k+1},x_{n-k+2}, ..., x_{n-1},x_n)
179: \end{equation}
180: with a specific precursory structure
181: \begin{equation}
182: {\mathbf x}^{pre}= (x_{n-k+1}^{pre}, x_{n-k+2}^{pre}, ...,x_{n-1}^{pre}, x_n^{pre}).
183: \end{equation}
184: %
185: Once the precursory structure ${\mathbf x}_{pre}$ is determined, we give an
186: alarm for an event $Y_{n+1}=1$ when we find the precursory variable
187: ${\mathbf x}_{(n-k+1,n)}$ inside the volume
188: %
189: %\begin{widetext}
190: \begin{eqnarray}
191: V^{pre}(\delta,{\mathbf x}^{pre}) & = & \prod_{j=n-k+1}^{n} \left(x_j^{pre}-\frac{\delta}{2},x_j^{pre}+\frac{\delta}{2}\right). \label{vol}
192: \end{eqnarray}
193: %\end{widetext}
194: %
195: There are different strategies to identify suitable precursory structures.
196: %
197: We choose the precursor via maximizing a conditional probability which we refer to as the {\sl likelihood} \cite{likelihood}.
198: %
199: \footnote{In this contribution we use the name likelihood for the probability that an event follows a precursor ${\mathbf x}$.
200: %
201: And the term {\sl aposterior pdf} for the probability to find a precursor ${\mathbf x}$ before of an already observed extreme event.
202: %
203: Note that the names might be also used vice versa, if one refers to the precursor as the previously observed information.}
204: %
205: The likelihood
206: \begin{eqnarray}
207: L(Y_{n+1}=1|{\mathbf x}_{(n-k+1,n)}) = \frac{j(Y_{n+1}=1,{\mathbf
208: x}_{(n-k+1,n)})}{\rho({\mathbf x}_{(n-k+1,n)})} \label{likely}
209: \end{eqnarray}
210: provides the probability that an event $Y_{n+1}=1$ follows the precursor ${\mathbf x}_{(n-k+1,n)}$.
211: %
212: It can be
213: calculated numerically by using the joint PDF $j((Y_{n+1}=1),{\mathbf
214: x}_{(n-k+1,n)})$.
215: %
216: Our prediction strategy consists of determining those values of each
217: component $x_i$ of ${\mathbf x}_{(n-k+1,n)}$ for which the likelihood is
218: maximal.
219:
220: This strategy to identify the optimal precursor represents a rather fundamental choice.
221: %
222: In more applied examples one looks for precursors which minimize or maximize
223: more sophisticated quantities, e.g., discriminant functions or loss matrices.
224: %
225: These quantities are usually functions of the posterior PDF or the
226: likelihood, but they take into account the additional demands of the
227: specific problem, e.g., minimizing the loss due to a false prediction.
228: %
229: The strategy studied in this contribution is thus fundamental in the
230: sense that it enters into many of the more sophisticated quantities which
231: were used for predictions and decision making.
232: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
233: \subsection{Testing for predictive power\label{roc}}
234: A common method to verify a hypothesis or to test the quality of a prediction
235: is the receiver operating characteristic curve (ROC) \cite{Swets1, Egans, Pepe}.
236: %
237: The idea of the ROC consists simply of comparing the rate of correctly
238: predicted events $r_{c}$ with the rate of false alarms $r_{f}$ by plotting
239: $r_c$ vs.\ $r_f$.
240: %
241: The rate of correct predictions $r_c$ and the rate of false
242: alarms $r_f$ can be obtained by integrating the {\sl aposterior} PDFs $\rho({\mathbf
243: x}_{(n-k+1,n)}|Y_{n+1}=1)$ and $\rho({\mathbf x}_{(n-k+1,n)}|Y_{n+1}=0)$ on the
244: precursory volume.
245: %
246: \begin{eqnarray}
247: r_c(\delta,{\mathbf x}^{pre}) & = & \int \rho({\mathbf x}_{(n-k+1,n)}|Y_{n+1}=1)
248: dV^{pre}(\delta,{\mathbf x}^{pre}) \nonumber
249: \label{rcor}\\
250: \\
251: r_f(\delta,{\mathbf x}^{pre}) & = & \int \rho({\mathbf x}_{(n-k+1,n)}|Y_{n+1}
252: =0) dV^{pre}(\delta,{\mathbf x}^{pre})\nonumber\\
253: \label{rf}
254: \end{eqnarray}
255: %
256: Note that these rates are defined with respect to the total
257: numbers of events $Y_{n+1}=1$ and nonevents $Y_{n+1}=0$.
258: %
259: Thus the relative frequency of events has no direct influence on the ROC, unlike on other measures of predictability, as e.g., the Brier score or the ignorance\cite{bandi}.
260:
261: Plotting $r_c$ vs $r_f$ for increasing values of $\delta$ one obtains a curve
262: in the unit-square of the $r_f$-$r_c$ plane (see, e.g., Fig.\ \ref{fig:rocgauss}).
263: %
264: The curve approaches the origin for $\delta \rightarrow0$ and the point $(1,1)$ in the limit $\delta \rightarrow \infty$, where $\delta$ accounts for the magnitude of the precursory volume $V_{pre}(\delta)$.
265: %
266: The shape of the curve characterizes the significance of the prediction. A curve above the diagonal reveals that the corresponding strategy of prediction
267: is better than a random prediction which is characterized by the
268: diagonal. Furthermore we are interested in curves which converge as fast as
269: possible to $1$, since this scenario tells us that we reach the highest
270: possible rate of correct prediction without having a large rate of false
271: alarms.
272:
273: That is why we use the so-called {\it likelihood ratio} as a summary index, to
274: quantify the ROC.
275: %
276: For our inference problems the likelihood ratio
277: is identical to the slope $m$ of the ROC-curve at the vicinity of the origin
278: which implies $\delta \rightarrow 0 $.
279: %
280: This region of the ROC is in particular interesting, since it corresponds to a low rate of false alarms.
281: %
282: The term likelihood ratio results from signal detection theory.
283: %
284: In the context of signal detection theory, the term {\sl a posterior PDF} refers to the PDF, which we call likelihood in the context of predictions and vice versa.
285: %
286: This is due to the fact that the aim of signal detection is to identify a
287: signal which was already observed in the past, whereas predictions are made
288: about future events.
289: %
290: Thus the \lq\lq likelihood ratio" is in our notation a ratio of a posterior PDFs.
291: %
292: \begin{eqnarray}
293: m & = & \frac{\Delta r_c}{\Delta r_f} \sim \frac{\rho({\mathbf x}^{pre}|Y_{n+1}=1)}{\rho({\mathbf x}^{pre}|Y_{n+1}=0)} + \mathcal{O}(\delta)\label{defm}.
294: \end{eqnarray}
295: %
296: However, we will use the common name likelihood ratio throughout the text.
297: For other problems the name likelihood ratio is also used for the slope at every point of the ROC.
298: %
299: Since we apply the likelihood ratio as a
300: summary index for ROC, we specify, that for our purposes the term likelihood ratio
301: refers only to the slope of the ROC curve at the vicinity of the
302: origin as in Eq.\ (\ref{defm}).
303:
304: %
305: Note, that one can show that the precursor, which maximizes the
306: likelihood as explained in Sec.\ \ref{pre} also maximizes the $m$ and is in
307: this sense the optimal precursor.
308: %
309: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%5
310: \subsection{Addressing the dependence on the event magnitude \label{evsize}}
311: We are now interested in learning how the predictability depends on the event magnitude
312: $\eta$ which is measured in units of the standard deviation of the time
313: series under study.
314: %
315: Thus the event variable $Y_{n+1}$ becomes dependent on the event
316: magnitude
317: \begin{eqnarray}
318: Y_{n+1}(\eta) & = &\left\{ \begin{array}{ll}
319: 1\quad&\mbox{ \small an event of magnitude $\eta$ or larger }\\
320: & \mbox{ occurred at time $n+1$} \\
321: 0\quad&\mbox{ \small no event of magnitude $\eta$ or larger}\\
322: &\mbox{ occurred at time $n+1$}\end{array} \right.
323: \end{eqnarray}
324:
325: Via Bayes' theorem the likelihood ratio can be
326: expressed in terms of the likelihood $L\bigl(Y_{n+1}(\eta)=1|{\mathbf
327: x}_{pre}\bigr)$ and the total probability to find events
328: $P\bigl(Y_{n+1}(\eta)=1\bigr)$.
329: %
330: Inserting the technical details of the calculation of the likelihood and the
331: total probability (see the appendix) we can see that the likelihood
332: ratio depends sensitively on the joint PDF
333: $j(\mathbf{x}_{(n-k+1,n)},Y_{n+1}(\eta)=1)$ of pecursory variable and event.
334: %
335: \begin{widetext}
336: \begin{eqnarray}
337: m (Y_{n+1}(\eta),\mathbf{x}_{(n-k+1,n)})& = &\frac{\left(1 - \int_{-\infty}^{\infty} d\mathbf{x}_{(n-k+1,n)} \; j(\mathbf{x}_{(n-k+1,n)},Y_{n+1}(\eta)=1)\right)}{\int_{-\infty}^{\infty} d\mathbf{x}_{(n-k+1,n)} \; j(\mathbf{x}_{(n-k+1,n)},Y_{n+1}(\eta)=1)}
338: %\times \nonumber\\
339: %&\it{}&
340: \frac{ \frac{j(\mathbf{x}_{(n-k+1,n)},Y_{n+1}(\eta)=1)}{\rho(\mathbf{x}_{(n-k+1,n)})}
341: }{\left(1 -
342: \frac{j(\mathbf{x}_{(n-k+1,n)},Y_{n+1}(\eta)=1)}{\rho(\mathbf{x}_{(n-k+1,n)})}\right)} \label{mgeneral}, \nonumber \\
343: \mbox{with} &\it{} & j(\mathbf{x}_{(n-k+1,n)},Y_{n+1}(\eta)=1) =
344: \int_{\mathcal{M}} \; dx_{n+1}\;
345: j(\mathbf{x}_{(n-k+1,n)},x_{n+1}),\quad \quad \nonumber\\
346: & \it{}& \mathcal{M} = \{ x_{n+1}: Y_{n+1}=1 \},\\
347: \mbox{and} & \it{}&\quad \rho(\mathbf{x}_{(n-k+1,n)}) =
348: j(\mathbf{x}_{(n-k+1,n)},Y_{n+1}(\eta)=1) +
349: j(\mathbf{x}_{(n-k+1,n)},Y_{n+1}(\eta)=0). \nonumber
350: \end{eqnarray}
351: \end{widetext}
352: %
353: Hence once the precursor is chosen, the dependence on the event
354: magnitude $\eta$ enters into the likelihood ratio, via the joint PDF of event and precursor.
355: %
356: Looking at the rather technical formula in Eq.\ (\ref{mgeneral}), there are
357: two aspects, which we find remarkable:
358: \begin{itemize}
359: \item[(I)] The slope of the ROC curve is fully characterized by the knowledge of
360: the joint PDF of precursory variable and event.
361: %
362: This implies that in the framework of
363: statistical predictions all kinds of (long-range) correlations which might be
364: present in the time series influence the quality of the predictions
365: only through their influence on the joint PDF.
366: %
367: \item[(II)] The definition of the event, e.g., as a threshold crossing or an
368: increment does change this dependence only insofar as it enters into the
369: choice of the
370: precursor and it influences also the set on which the integrals in Eq.\ (\ref{mgeneral}) are
371: carried out.
372: %
373: Both $Y_{n+1}(\eta)$ and the set $\mathcal{M}$ have
374: to be defined according to the type of events one predicts.
375: %
376: When predicting, e.g., increments $x_{n+1} - x_n \geq \eta $ via the precursory variable $x_n$, then $\mathcal{M}=[a,b]$ with
377: $ a(Y_{n+1}(\eta)) = x_n + \eta$ for the lower border and $ b(Y_{n+1}(\eta)) =
378: \infty$ for the upper border.
379: %
380: In order to predict threshold crossings at $x_{n+1}$ via $x_n$ one uses $ a(Y_{n+1}(\eta)) = \eta$, $ b(Y_{n+1}(\eta)) = \infty$.
381: \end{itemize}
382: %
383:
384: Exploiting Eq.\ (\ref{mgeneral}) we can hence determine the dependence of the
385: likelihood ratio and the ROC curve on the events magnitude $\eta$, via the
386: dependence of the joint PDF of the process under study.
387: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
388: \subsection{Constraint for increasing quality of predictions with increasing
389: event magnitude \label{constr}}
390: %
391: In order to study the dependence of the likelihood ratio on the event magnitude we are going to introduce a constraint which the likelihood and the total
392: probability to find events have to fulfill in order to find a better
393: predictability of larger (smaller) events.
394:
395: In order to improve the readability of the paper, we will first introduce
396: the following notations for %
397: the aposterior PDFs, the likelihood and the total probability to find events
398: \begin{eqnarray}
399: \rho_c({\mathbf x}_{(n-k+1,n)}, \eta) & = & \rho({\mathbf
400: x}_{(n-k+1,n)}|Y_{n+1}(\eta)=1), \; \\
401: \rho_f ({\mathbf x}_{(n-k+1,n)}, \eta) & = & \rho({\mathbf
402: x}_{(n-k+1,n)}|Y_{n+1}(\eta)=0),\; \\
403: L (\eta, {\mathbf x}_{(n-k+1,n)})& = & L(Y_{n+1}(\eta)=1|{\mathbf
404: x}_{(n-k+1,n)}), \;\\
405: P(\eta) & = & P(Y_{n+1}(\eta)=1).
406: \end{eqnarray}
407: We can then ask for the change of the
408: likelihood ratio with changing event magnitude $\eta$.
409: \begin{equation}
410: \frac{\partial}{\partial \eta}\, m(Y_{n+1}(\eta),\mathbf{x}_{(n-k+1,n)}) \gtreqqless0.
411: \end{equation}
412: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
413: The derivative of the likelihood ratio is positive (negative, zero), if the following sufficient condition $c(\eta)$ is fulfilled.
414: \begin{widetext}
415: \begin{equation}
416: c(\eta, {\mathbf x}_{(n-k+1,n)}) = \frac{\partial}{\partial \eta}\ln L(\eta,{\mathbf x}_{(n-k+1,n)})-
417: \frac{\bigl(1-L(\eta,{\mathbf x}_{(n-k+1,n)}))\bigr)}{\bigl(1-P(\eta)\bigr)}\;\frac{\partial}{\partial
418: \eta}\ln P(\eta) \gtreqqless 0. \label{c2b}
419: \end{equation}
420: \end{widetext}
421: Hence one can tell for an arbitrary process, if extreme events are better
422: predictable, by simply testing, if the marginal PDF of the event and the
423: likelihood of event and precursor fulfill Eq.\ (\ref{c2b}).
424:
425: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
426: \section{Predictions of Increments in i.i.d.\ random numbers \label{num}}
427: %
428: In this section we test the condition $c(\eta,{\mathbf x}_{(n-k+1,n)})$ as given in Eq.\ (\ref{c2b}) for increments in Gaussian, power-law,
429: and exponentially distributed i.i.d.\ random numbers.
430: %
431: We thus concentrate on extreme events which consist of a sudden increase (or decrease) of the observed variable within a
432: few time steps.
433: %
434: Examples of this kind of extreme events are the increases in
435: wind speed in \cite{Physa,Euromech}, but also stock market
436: crashes \cite{stocks,Sornette2} which consist of sudden decreases.
437:
438: We define our extreme event by an increment $x_{n+1}-x_n$
439: exceeding a given threshold $\eta$
440: %
441: \begin{equation}
442: x_{n+1}-x_n \geq \eta, \label{e0}
443: \end{equation}
444: %
445: where $x_{n}$ and $x_{n+1}$ denote the observed values at two consecutive time
446: steps and the event magnitude $\eta$ is again measured in units of the standard deviation.
447:
448: Since the first part of the increment $x_n$ can be used as a precursory variable, the definition of the event as an increment introduces a correlation between the event and the precursory variable
449: $x_n$.
450: %
451: Hence the prediction of increments in random numbers provides a simple, but not unrealistic example which allows us to study the influence of the
452: distribution of the underlying process on the event-magnitude dependence of the
453: quality of prediction.
454:
455: In the examples which we study in this section the joint PDF of
456: precursory variable and event is known and we can hence evaluate $c(\eta,x_n)$
457: analytically.
458: %
459: A mathematical expression for a filter which selects the PDF of our
460: extreme events out of the PDFs of the underlying stochastic process can be
461: obtained through applying the Heaviside function $ \Theta( x_{n+1} - x_{n}
462: -\eta)$ to the joint PDF.
463: %
464: This method is described in more detail in the appendix.
465:
466: Since in most cases the structure of the PDF is not known analytically, we are also
467: interested in evaluating
468: $c(\eta,x_n) $ numerically.
469: %
470: In this case the approximations of the total probability and the likelihood are obtained by \lq\lq binning and counting" and
471: their numerical derivatives are evaluated via a
472: Savitzky-Golay filter \cite{Savgol,numrecipes}.
473: %
474: The numerical evaluation is done within $10^7$ data points. In order to check
475: the stability of this procedure, we evaluate $c(\eta,x_n)$ also on $20$
476: bootstrap samples which are generated from the original data set.
477: %
478: These bootstrap samples consist of $10^7$ pairs of event and precursory variable, which were drawn randomly from the
479: original data set.
480: %
481: Thus their PDFs are slightly different in their first and second
482: moment and they contain different numbers of events.
483: %
484: Evaluating $c(\eta, x_n)$ on the bootstrap samples thus shows how
485: sensitive our numerical evaluation procedure is towards changes in the
486: numbers of events. This is especially important for large and therefore rare
487: events.
488:
489: In order to check the results obtained by the evaluation of $c(\eta,x_n) $, we
490: compute also the corresponding ROCs analytically and numerically.
491:
492: Note that for both, the numerical evaluation of the condition and the
493: ROC curves, we used only
494: event magnitudes $\eta$ for which we found at least $1000$ events, so that the
495: observed effects are not due to a lack of statistics of the large events.
496: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
497: \subsection{Gaussian distributed random numbers \label{Gaussian}}
498: %%%%%%%%%%%%%%%
499: \begin{figure}
500: \epsfig{file=figure1.eps,width=8cm}
501: \caption{The condition $c(\eta,x_n)$ for the
502: Gaussian distribution as given by Eq.\ (\ref{cgauss}).
503: %
504: The color shaded regions indicate the intervals $[-\sigma \eta, -\eta/2]$ for
505: which we can according to Eq.\ \ref{gausseins} expect
506: $c(\eta,x_n)$ to be positive.
507: %
508: If $x_n < -\sigma \eta $, $\eta >
509: 2\sqrt{\pi}$ and terms of the order of $\exp(-(x_n + \sigma \eta)^2)$ are
510: sufficiently small, the condition is also positive.
511: %
512: If terms of the order of $\exp(-(x_n + \sigma \eta)^2)$ cannot be neglected one also might find
513: small regions in $(-\infty, -\sigma \eta]$ for which $c(\eta,x_n) <0$.
514: %
515: However, the influence of these regions
516: is neglectable, since our
517: alarm interval is defined as $[-\infty,\delta]$ which implies an averaging
518: over several possible values of the precursory variable. \label{fig:cgauss}
519: }
520: \end{figure}
521: %%%%%%%%%%%%%%%%
522: In the first example we assume the sequence of i.i.d.\ random numbers which
523: form our time series to be normal distributed.
524: %
525: As we know from \cite{Sarah2}, increments
526: within Gaussian random numbers are better predictable, the more extreme
527: they are.
528: %
529: In this section we will show that their PDFs fulfill also the
530: condition in Eq.\ (\ref{c2b}).
531: Applying the filter mechanism developed in the appendix we obtain the following expressions for the a posteriori PDFs
532: \begin{equation}
533: \rho_c(x_n,\eta) = \frac{\exp\left(-\frac{x_{n}^2}{2\sigma^2}\right)}{2\sqrt{2\pi}\sigma
534: P(\eta)}\mbox{erfc}\left(\frac{x_{n}+\sigma\eta}{\sigma\sqrt{2}}\right),\label{marga}
535: \end{equation}
536: and the likelihood
537: \begin{equation}
538: L(\eta,x_n) = \frac{1}{2}\erfc\left(\frac{x_n +
539: \sigma\eta}{\sigma\sqrt{2}}\right). \label{gausslikely}
540: \end{equation}
541: %
542: We recall that the optimal precursor is given by the value of $x_n$ which
543: maximizes the likelihood. %
544: We refer to this special value of the variable $x_n$ by $x_{pre}$ and find for
545: the likelihood according to Eq.\ (\ref{gausslikely}) $x_{pre}=-\infty$.
546: %
547: Thus, instead of a finite alarm volume $\delta$
548: here is the upper limit of the interval $[-\infty, \delta ]$.
549: %
550: %
551: The total probability to find increments of magnitude $\eta$ is given by
552: %
553: \begin{equation}
554: P(\eta) = \frac{1}{2} \erfc(\eta/2).\label{gaussptotal}
555: \end{equation}
556: %%%%%%%%%%%%%%%
557: \begin{figure}
558: \epsfig{file=figure2.eps ,width=8cm}
559: \caption{Comparison of the numerically
560: evaluated condition $c(\eta,x_n)$ for the Gaussian
561: distribution and the expression given by Eq.\ (\ref{cgauss}).
562: %
563: The black curves denote the evaluation of the analytic result in Eq.\
564: (\ref{cgauss}), the curves plotted with lines and symbols represent the numerical results
565: obtained from the original data set, and the dashed lines represent the
566: results obtained from the corresponding bootstrap samples.
567: %
568: The gray (green in the colored plot) regions indicate the regime $-\sigma \eta
569: < x_n < -\sigma \eta /2$ for which $c(\eta, x_n)$ is positive in the limit
570: $\eta \rightarrow \infty$.
571: %
572: The numerical
573: evaluation of $c(\eta,x_n)$ was done by sampling the likelihood and the total
574: probability of events from $10^7$ random numbers.\label{fig:cnumgauss}}
575: \end{figure}
576: %%%%%%%%%%%%%%%%
577: Hence the condition in Eq.\ (\ref{c2b}) reads
578: \begin{eqnarray}
579: c(\eta,x_n) & = &-\sqrt{\frac{2}{\pi}}\frac{\exp\left(-z^2\right)}
580: {\erfc\left(z\right)} \quad \nonumber\\
581: &\it{}& + \frac{1}{\sqrt{\pi}} \frac{\exp\left(-\frac{\eta^2}{4}\right)}{\erfc\left(\frac{\eta}{2}\right)}
582: \frac{\left(1 -\frac{1}{2}\erfc\left(z\right) \right)}
583: {\left( 1- \frac{1}{2} \erfc\left(\frac{\eta}{2}\right)\right)}, \nonumber\\
584: &\it{}& \mbox{with}\quad z=\frac{x_n + \sigma \eta}{\sqrt{2}\sigma} \label{cgauss}
585: \end{eqnarray}
586: Fig.\ \ref{fig:cgauss} illustrates this expression
587: and Figure \ref{fig:cnumgauss} compares it to the numerical results.
588: %
589: For the ideal precursor $x_{n} = x_{pre} = -\infty $ the condition $c(\eta,x_n)$ is
590: ---according to Eq.\ (\ref{cgauss})---
591: zero, since in this case,
592: the slope of the ROC-curve tends to infinity \cite{Sarah2} and does not react to any
593: variation in $\eta$.
594: %
595: For any finite value of the precursory variable $x_{n} <0$ we have to distinguish three regimes
596: of $z=(x_n + \sigma \eta)/\sqrt{2}\sigma$, namely, $z \rightarrow \infty$ or $z
597: \rightarrow -\infty$ and finally also the case $z=0$.
598:
599:
600: In the first case we study the
601: behavior of $c(\eta,x_n)$ for a fixed value of the precursory variable $-\sigma \eta <x_n$ and $\eta
602: \rightarrow \infty$.
603: %
604: This implies that $z\rightarrow \infty$ and we can use
605: the asymptotic expansion for large arguments of the complementary
606: error function
607: %%%%%%%%%%%%
608: \begin{figure}[t!!!]
609: \includegraphics[width=9cm]{figure3.eps}
610: \caption[]{\small\label{fig:rocgauss} ROCs for Gaussian distributed i.i.d.\ random variables.
611: %
612: The symbols
613: represent ROC curves which where made via predicting increments in $10^7$ normal
614: i.i.d.\ random numbers.
615: %
616: The predictions were made according to the prediction
617: strategy described in Sec.\ \ref{pre}. The lines represent the results of evaluating
618: the integrals in Eqs.\ (\ref{rcor}) and (\ref{rf}) for the Gaussian case. Note that the quality of the
619: prediction increases with increasing event magnitude.}
620: \end{figure}
621: %%%%%%%%
622: %
623: \begin{eqnarray}
624: \erfc(z) &\sim & \frac{\exp(-z^2)}{\sqrt{\pi}z} \left(1 + \sum_{m=1}^{\infty}
625: (-1)^m \frac{1 \cdot 3 ...(2m-1)}{(2z^2)^m }\right),\nonumber\\
626: &\it{}& \left( z \rightarrow \infty, |
627: \mbox{arg} z | < \frac{3\pi}{4} \right)\label{erfcapprox}
628: \end{eqnarray}
629: %
630: which can be found in \cite{Abram} to obtain
631: %
632: \begin{eqnarray}
633: c(\eta, x_n) & \propto & -\frac{x_n}{\sigma} + \frac{\eta}{2},\quad -\sigma \eta < x_n <0. \label{gausseins}
634: \end{eqnarray}
635: %
636: This expression is appropriate for $x_n > -\sigma \eta$ since the
637: asymptotic expansion in Eq.\ (\ref{erfcapprox}) holds only if the argument of
638: the complementary error function is positive.
639: %
640: In this case $c(\eta, x_n)$ is larger than zero, if $x_n$ is fixed and finite and $-\sigma \eta < x_n < -\sigma \eta/2$.
641: %
642:
643: In the second case, we assume $\eta \gg 1$ to be fixed, $x_n <-\sigma \eta$ and $x_n
644: \rightarrow -\infty$.
645: %
646: Hence we can use the expansion in Eq. (\ref{erfcapprox}) only to obtain
647: the asymptotic behavior of the dependence on $\eta$ and not for the dependence
648: on $z$.
649: %
650: An asymptotic expression of $c(\eta, x_n)$ hence reads
651: %
652: \begin{eqnarray}
653: c(\eta, x_n) & \propto & \frac{\eta}{2\left(1 - \frac{1}{2}
654: \erfc\left(\eta/2\right) \right)} \left(\frac{\erf(z)}{\sqrt{pi}} +
655: \frac{\eta}{2} \right) \nonumber\\
656: &\it{}& - \mathcal{O}\left(\exp(-z^2) \right), \quad x_n < -\sigma
657: \eta. \label{gausszwei}
658: \end{eqnarray}
659: %
660: Since $\erf(z)$ tends to minus unity as $z \rightarrow -\infty$ the expression
661: in Eq. (\ref{gausszwei}) is positive if $\eta > 2\sqrt{\pi}$ and if we can
662: assume the squared exponential term to be sufficiently small.
663: %
664: If the later assumption is not fullfilled one might observe some regions of
665: intermediate values of $-\infty< x_n < -\sigma \eta$, for which $c(\eta,x)$ is
666: negative.
667:
668: However the ROC curves in Fig.\ \ref{fig:rocgauss} suggest that the influence
669: of these regions is sufficiently small, if the alarm volume is chosen to be $[-\infty, \delta ]$.
670: %
671: We can understand this effect, if we keep in mind that we use the interval
672: $[-\infty, \delta]$ as an alarm volumen.
673: %
674: Hence we can expect that the influence of the regions, where $c(\eta, x_n)$ is negative, is suppressed since we average over many different values of $x_n$ and the condition is positive as $x_n \rightarrow -\infty$.
675: %
676: (Positive is meant here
677: in the sense, that $c(\eta,x_n)$ approaches the value zero for $x_n = -\infty$
678: from small positive numbers.)
679:
680:
681: In the third case, for $x_n = -\sigma \eta$ and hence $z=0$ we find that $c(\eta,x_n)$ is
682: positive if $\eta > 2\sqrt{\frac{2}{\pi}} \left(1 - \frac{1}{2}\erfc(\eta/2) \right)$.
683: %
684:
685: In total we can expect larger increments in Gaussian random numbers to be easier
686: to predict the larger they are.
687: %
688: The ROCs in Fig.\ \ref{fig:rocgauss} support these results.
689:
690: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
691:
692: \subsection{Symmetrized exponential distributed random variables
693: \label{symexpo}}
694: The PDF of the symmetrized exponential reads
695: \begin{eqnarray}
696: \rho(x) & = & \frac{\lambda}{2} \exp(-\lambda |x_n|) = \left\{ \begin{array}{l@{\quad:\quad}l} \frac{\lambda}{2} \exp(-\lambda x_n) & x_n
697: >0, \\ \lambda/2 & x_n =0,\\ \frac{\lambda}{2} \exp(\lambda x_n) & x_n
698: <0, \end{array} \right. \nonumber
699: \end{eqnarray}
700: %%the factor $\frac{1}{2}$ comes from normalization over the whole real axis.
701: with $\mu =0$, $\sigma = \sqrt{2}/\lambda$.
702:
703: Applying the filtering mechanism
704: according to the appendix we find the joint PDFs of precursory variable and event
705: \begin{widetext}
706: \begin{eqnarray}
707: j(x_n,(Y_{n+1}(\eta)=1)) & = & \left\{ \begin{array}{l@{\quad :\quad}l}
708: \frac{\lambda}{4}\exp(-\sqrt{2} \eta -2\lambda x_n) & x_n >0,\\
709: \frac{\lambda}{4}\exp(-\sqrt{2}\eta) & -\eta <x_n <0,\\
710: \frac{\lambda}{2} \left(\exp(\lambda x_n) -\frac{1}{4}\exp\left(\sqrt{2}\eta
711: +\lambda 2x_n \right) \right) & x_n < - \eta <0,\\
712: \end{array} \right.
713: \end{eqnarray}
714: \end{widetext}
715: %
716: \begin{widetext}
717: the aposterior probabilities,
718: \begin{eqnarray}
719: \rho_c(x_n, \eta, \lambda) & = & \left\{\begin{array}{l@{\quad :\quad}l}
720: \frac{\lambda}{(2 + \sqrt{2}\eta}\exp( -2\lambda x_n) & x_n >0,\\
721: \frac{\lambda}{(2+\sqrt{2}\eta}& -\eta <x_n <0,\\
722: \frac{\lambda}{(2+ \sqrt{2}\eta} \left(2\exp(\sqrt{2}\eta + \lambda x_n) -\exp\left(2\sqrt{2}\eta
723: +2\lambda x_n \right) \right) & x_n < - \eta <0,\\
724: \end{array} \right. \label{symexpapostc}\\
725: %
726: \rho_f(x_n,\eta, \lambda) & = & \left\{\begin{array}{l@{\quad :\quad}l}
727: \frac{\lambda}{2}\exp( -\lambda x_n) \frac{\left(1 - \frac{1}{2}
728: \exp(-\lambda x_n - \sqrt{2}\eta) \right)}{\left(1 -
729: \frac{1}{2}\left( 1 + \frac{\eta}{2}\right) \exp(-\sqrt{2}\eta) \right)} & x_n >0,\\
730: \frac{\lambda}{2}\exp( \lambda x_n) \frac{\left(1 - \frac{1}{2}
731: \exp(-\lambda x_n - \sqrt{2}\eta) \right)}{\left(1 -
732: \frac{1}{2}\left( 1 + \frac{\eta}{2}\right) \exp(-\sqrt{2}\eta) \right)} & -\eta <x_n <0,\\
733: \frac{\lambda}{4} \frac{\exp(2\lambda x_n + \sqrt{2}\eta)}{\left(1 -
734: \frac{1}{2}\left( 1 + \frac{\eta}{2}\right) \exp(-\sqrt{2}\eta) \right)} & x_n < - \eta <0,\\
735: \end{array} \right. \label{symexpapostf}
736: \end{eqnarray}
737: \end{widetext}
738: \begin{widetext}
739: the likelihood
740: \begin{eqnarray}
741: L(\eta,x_n, \lambda) & = & \left\{ \begin{array}{l@{\quad :\quad}l}
742: \frac{1}{2}\exp(-\sqrt{2} \eta -\lambda x_n) & x_n >0,\\
743: \frac{1}{2}\exp(-\sqrt{2}\eta - \lambda x_n) & -\eta <x_n <0,\\
744: 1 -\frac{1}{2}\exp(\sqrt{2}\eta +\lambda x_n ) & x_n < - \eta <0,\\ \end{array} \right. \label{symexplike}
745: \end{eqnarray}
746: \end{widetext}
747: %
748: and the total probability to find events of magnitude $\eta$
749: %\begin{widetext}
750: \begin{eqnarray}
751: P(\eta) & = & \left\{ \begin{array}{l@{\quad :\quad}l}
752: \frac{1}{8}\exp(-\sqrt{2} \eta) & x_n >0,\\
753: \frac{\sqrt{2}}{4}\eta \exp(-\sqrt{2}\eta) & -\eta <x_n <0,\\
754: \frac{3}{8}\exp(-\sqrt{2}\eta) & x_n < - \eta <0. \end{array}\right. \quad
755: \end{eqnarray}
756: %\end{widetext}
757: If we are not interested in the range of the precursory variable, the total probability
758: to find events is given by
759: \begin{equation}
760: P(\eta) = \frac{1}{2}\exp(-\sqrt{2}\eta) \left(1 + \frac{\eta}{\sqrt{2}} \right).
761: \end{equation}
762: %
763: Hence the condition $c(\eta, x_n, \lambda )$ reads
764: \begin{widetext}
765: %
766: %%%%%%%%%%%%%
767: \begin{figure*}[t!!!]
768: \parbox{18cm}{
769: %\setlength{\fboxsep}{}
770: \parbox{8.5cm}{
771: \epsfig{file=figure4.eps, width=8cm}
772: \caption{The numerically and analytically evaluated condition
773: for the symmetrized exponential. The black line is the result of the
774: analytical evaluation according to Eq.\ (\ref{csymexp}), the
775: curves plotted with lines and symbols represent the numerical results
776: obtained from the original data set, and the dashed lines represent the
777: results obtained from the corresponding bootstrap samples. Note that for
778: small values of $x_n$ the condition $c(\eta, x_n, \lambda)$ is for all values of $\eta$ close
779: to zero.\label{fig:csymexp}}}
780: %\end{figure}
781: %%%%%%
782: \hspace*{0.5cm}
783: %%%%%%%%%%%%%
784: %\begin{figure}[t!!!]
785: \parbox{8.5cm}{
786: \epsfig{file=figure5.eps,width=8cm}
787: \caption{The ROCs for symmetrically exponentially
788: distributed i.i.d.\ random numbers show no significant dependence on the
789: event magnitude. The ROC curves were made via
790: predicting increments in $10^7$ normal i.i.d.\ random numbers and the predictions were made according to the prediction
791: strategy described in Sec.\ \ref{pre}. The black
792: line indicates the analytically evaluated ROC curve for
793: $\eta=0$. \label{fig:rocsymexp}}
794: }
795: }
796: \end{figure*}
797: %%%%%%%%%%%%%%
798: \begin{eqnarray}
799: c(\eta, x_n,\lambda) & = & \left\{\begin{array}{l}
800: -\sqrt{2}\left(1 - \frac{\left(1 - \frac{1}{2} \exp(-\sqrt{2}\eta - \lambda x_n)\right)}{\left(1 - \frac{1}{8}\exp(-\sqrt{2}\eta)\right)}\right), \quad x_n >0,\\
801: \\
802: -\sqrt{2} + \frac{(1 -\sqrt{2}\eta)}{\eta} \frac{\left(1- \frac{1}{2}\exp(-\sqrt{2}\eta
803: -\lambda x_n)\right)}{\left(1 -
804: \frac{\sqrt{2}}{4}\eta \exp(-\sqrt{2}\eta)\right)}, \quad -\eta<x_n<0,\\
805: \\
806: -\frac{1}{\sqrt{2}}\exp(\lambda x_n + \sqrt{2}\eta)\left(\frac{1}{1 -
807: \frac{1}{2}\exp(\sqrt{2}\eta + \lambda x_n)} + \frac{1}{1 -
808: \frac{3}{8}\exp(-\sqrt{2}\eta)} \right),\quad x_n < - \eta.
809: \end{array}\right. \label{csymexp}
810: \end{eqnarray}
811: \end{widetext}
812:
813: Figure \ref{fig:csymexp} compares the
814: results of the numerical evaluation of the condition and the analytical
815: expression given by Eq.\ (\ref{csymexp}). Since most precursors of large
816: increments can be found among negative values, the numerical evaluation of
817: $c(\eta,x_n\lambda)$ becomes worse for positive values of $x_n$, since in
818: this limit the likelihood is not very well sampled from the data. This leads
819: also to the wide spread of the bootstrap samples in this region.
820:
821: Figure \ref{fig:csymexp} shows that in the vicinity of the smallest value of the data set, the condition
822: $c(\eta,x_n,\lambda)$ is zero. As we approach larger values of $\eta$,
823: $c(\eta,x_n,\lambda)$ approaches zero in the whole range of data values. That
824: is why we would expect to see no influence of the event magnitude on the quality of predictions in the exponential case.
825:
826: The ROC curves in Fig. \ref{fig:rocsymexp} support these results. The
827: numerical ROC curves were made via predicting increments in $10^7$ normal
828: i.i.d.\ random numbers according to the prediction
829: strategy described in Sec. \ref{pre}.
830: %
831: The precursor for the ROC-curves is chosen as the maximum of the likelihood
832: according to Eq.\ (\ref{symexplike}), i.e., $x_{pre}=-\infty$, so that the
833: alarm interval is $[\infty,\delta]$.
834: %
835: In summary there is no significant dependence on the event magnitude for the prediction
836: of increments in a sequence of symmetrical exponential distributed random numbers.
837: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
838: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
839: \subsection{Pareto distributed random variables \label{powl}}
840: %%%%%%%%%%%%%%%%
841: \begin{figure}[t!!!]
842: \includegraphics[width=6cm,angle=-90]{figure6.eps}
843: \caption[]{\small The condition $c(\eta,x_n,x_{min},k)$ for the power-law distribution with lower endpoint
844: $x_{min} =0.01$ are plotted for constant values of the precursory variable
845: $x_{n}$. The symbols represent the results of the numerical evaluation of
846: $c(\eta,x_n,x_{min},k)$, the gray (colored) lines denote the analytic results,
847: and the black lines denote the result for the corresponding bootstrap samples
848: and the optimal precursor.
849: %
850: For the \lq\lq ideal" precursor $x_{n}=x_{min}=0.01$ all
851: values of $c(\eta,k, 0.01)$ are negative. Hence one should expect smaller
852: events to be better predictable. However, this effect is sensitive of the
853: choice of the precursor. \label{fig:powerlawcompare}}
854: %%%%%%%%%%%%%%%%}
855: \end{figure}
856: We investigate the Pareto distribution as an example for power-law
857: distributions.
858: %
859: The PDF of the Pareto distribution is defined as \cite{Feller}
860: \begin{equation}
861: \rho(x) = kx_{min}^k \, x^{-(k+1)}
862: \end{equation}
863: for $x \in [x_{min}, \infty)$ with the exponent $k\geq3$, the lower endpoint
864: $x_{min} > 0$, and variance $\sigma = \frac{x_{min}}{k-1}\sqrt{\frac{k}{k-2}}$.
865: Filtering for increments of magnitude $\eta$
866: we find the following conditional PDFs of the increments:
867: \begin{eqnarray}
868: \rho_c ( x_n, \eta, x_{min},k)& = & \frac{k x_{min} ^{2k}}{ x_n^{k+1} \left(x_n +
869: \frac{x_{min}}{k+1}\sqrt{\frac{k}{k-2}}\, \eta \right)^k P(\eta,k)}
870: \label{paretoapost}\nonumber\\
871: \\
872: %
873: \rho_f (x_n, \eta, x_{min},k)& = & \frac{kx_{min}^k}{x_n^{k+1}} \frac{\left(1 -
874: \left(\frac{x_{min}}{x_n + \frac{x_{min}}{k+1}\sqrt{\frac{k}{k-2}} \,\eta
875: }\right)^k\right)}{1 - P(\eta,k)}\nonumber\\
876: \\
877: %
878: L (\eta, x_n, x_{min},k)& = &\left( \frac{x_{min}}{x_n +
879: \frac{x_{min}}{k+1} \sqrt{\frac{k}{k-2} \, \eta}}\right)^k .
880: \label{paretolikeli}
881: \end{eqnarray}
882: Within the range $(x_{min}, \infty)$ the likelihood has no well defined
883: maximum. However, since the likelihood is a monotonously decreasing
884: function, we use the lower endpoint $x_{min}$ as a precursor.
885: The total probability to find events of magnitude $\eta$ is given by
886: %\begin{widetext}
887: \begin{eqnarray}
888: P(\eta,k)& = & \frac{1}{2} \,{}_{2}F_1
889: \left(k, 2k, 2k+1,- \frac{\eta}{(k+1)}\sqrt{\frac{k}{k-2}}\right), \nonumber \\\label{paretototal}
890: \end{eqnarray}
891: where $ {}_{2}F_1(a,b,c,x) $ denotes the hypergeometric function $ {p}_{2}F_q(a,b,c,x)$ with
892: $p=2$, $q=1$.
893: %\end{widetext}
894:
895: %%%%%%%%%%%%%%%%
896: \begin{figure}[t!!!]
897: \includegraphics[width=6cm, angle= -90]{figure7.eps}
898: \caption[]{\small\label{fig:powlroc3}ROC-plot for the power-law distribution with $k=3$ and $x_{min} =0.01$. The
899: symbols show the numerical results and the lines indicate the analytically
900: calculated ROC curves.
901: The ROC curves were made via predicting increments in $10^7$ Pareto
902: distributed i.i.d.\ random numbers. The predictions were made according to
903: the prediction strategy described in Sec.\ \ref{pre}. Note that we tested only
904: event magnitudes $\eta$, for which we found at least $1000$ events, so that the
905: effects we observe are not due to a lack of statistics of the large events. The ROC curves display
906: that in Pareto distributed i.i.d.\ random numbers with the lower endpoint
907: $x_{min}=0.01$ smaller events are better predictable and that large events are
908: very hard to predict.}
909: \end{figure}
910: %%%%%%%%%%%%%%%
911:
912: %%%%%%%%%%%%%%%%
913: %\begin{figure}[t!!!]
914: %\includegraphics[width=6cm, angle= -90]{figure8.eps}
915: %\caption[]{\small\label{fig:powlroc6} (Color online)
916: %ROC-plot for the power-law distribution with $k=6$ and $x_{min} =0.01$. The
917: %symbols show the numerical results, the lines indicate the analytically
918: %calculated ROC-curves.
919: %The ROC-curves where made via predicting increments in $10^7$ Pareto distribute%d
920: %i.~i.~d.\ random numbers. The predictions were made according to the prediction%
921: %strategy described in Sec.\ \ref{pre}. The ROC-curves show
922: %that in Pareto distributed i.~i.~d.\ random numbers with the lower endpoint
923: %$x_{min}=0.01$ smaller events are better predictable and that large events are %especially
924: %hard to predict. }
925: %\end{figure}
926: %%%%%%%%%%%%%%%
927:
928: Using
929: \begin{eqnarray}
930: \ \frac{\partial P(\eta, k)}{\partial \eta} & = & \frac{k}{\eta} \left(
931: \frac{1}{\left(1 + \frac{\eta}{k+1}\sqrt{\frac{k}{k-2}} \right)^k} - 2P(\eta,k)\right) \nonumber\\
932: \end{eqnarray}
933: and inserting the expressions (\ref{paretolikeli}) and (\ref{paretoapost}) for the components of
934: $c(\eta,x_n, x_{min},k)$ we can obtain an explicit analytic expression for
935: the condition. In Fig.\ \ref{fig:powerlawcompare} we evaluate this expression using {\it
936: Mathematica} and compare it with the results of an empirical evaluation on
937: the data set of $10^7$ i.i.d.\ random numbers.
938:
939: Figure (\ref{fig:powerlawcompare}) displays that the value
940: of $c(\eta,x_n, x_{min},k)$ depends sensitively on the choice of the
941: precursor.
942: %
943: For the ideal precursor $x_{pre}=x_{min}$ all values of $c(\eta,k, x_{min})$ are negative.
944: %
945: Hence one should in this case expect smaller
946: events to be better predictable.
947: %
948: The corresponding ROC curves in Figs.\
949: \ref{fig:powlroc3}, %%\ref{fig:powlroc6}
950: and \ref{fig:powlroc9} verify this
951: statement of $c(\eta,x_n, x_{min},k)$.
952:
953: In summary we find that larger events in Pareto distributed i.~i.~d.\ random
954: numbers are harder to predict the larger they are. This is an admittedly
955: unfortunate result, since extremely large events occur much more frequently in
956: power-law distributed processes than in Gaussian distributed processes. Hence,
957: their prediction would be highly desirable.
958: %%%%%%%%%%%%%%%%
959: \begin{figure}[t!!!]
960: \includegraphics[width=6cm, angle= -90]{figure9.eps}
961: \caption[]{\small\label{fig:powlroc9}ROC-plot for the power-law distribution with $k=9$ and $x_{min} =0.01$.
962: %
963: The symbols show the numerical results and the lines indicate the analytically
964: calculated ROC curves.
965: %
966: The ROC curves where made via predicting increments in $10^7$ Pareto distributed
967: i.i.d.\ random numbers and the predictions were made according to the prediction
968: strategy described in Sec.\ \ref{pre}.
969: %
970: The ROC-curves display that in Pareto distributed i.i.d.\ random numbers smaller events are better
971: predictable and large events are especially hard to predict.}
972: \end{figure}
973: %%%%%%%%%%%%%%%
974:
975: \section{Increments in Free Jet Data \label{freejet}}
976: In this section, we apply the method of statistical inference to predict
977: acceleration increments in free jet data.
978: %
979: Therefore we use a data set of
980: $1.25\times 10^7$ samples of the local velocity measured in the turbulent
981: region of a round free jet \cite{Peinke}.
982: %
983: The data were sampled by a hot-wire
984: measurement in the central region of an air into air free jet.
985: %
986: One can then calculate the PDF of velocity increments $a_{n,k}=v_{n+k} -v_n$, where $v_n$ and
987: $v_{n+k}$ are the velocities measured at time step $n$ and $n+k$. The Taylor
988: hypothesis allows one to relate the time-resolution to a spatial
989: resolution \cite{Peinke}.
990: %
991: One observes that for large values of $k$ the PDF of increments is essentially indistinguishable
992: from a Gaussian, whereas for small $k$, the PDF develops approximately
993: exponential wings \cite{vanAtta, Frisch1990, Frisch1995}.
994: %
995: Fig.\ \ref{fjhisto} illustrates this effect using the data set under study.
996: %%%%%%%%%%%%%%%%%%%%
997: \begin{figure}
998: \epsfig{file=figure10.eps,width=8cm}
999: \caption{\label{fjhisto}PDF of the increments $a_{n,k}=v_{n+k}-v_n$ with $k=
1000: 1,3,10,35,144,285$. The black lines correspond to Gaussian and exponential
1001: PDFs with appropriate values for the standard deviation or the coefficient $\lambda$.}
1002: \end{figure}
1003: %%%%%%%%%%%%%%%
1004: %%%%%%%%%%%%%%%%%%%%%%%
1005: \begin{figure*}[t!!!]
1006: \centerline{
1007: \epsfig{file=figure11.eps,width=8cm} \hspace{0.01cm}
1008: \epsfig{file=figure12.eps,width=8cm}}
1009: \caption[]{\label{fjcondi} Transition from the exponential regime
1010: (a) to the Gaussian regime (b) characterized via the numerical evaluation of
1011: $c(x_n,\eta)$. The black line corresponds to the analytic results for the
1012: Gaussian and the exponential PDF, fitted to the PDFs of the
1013: increments, as it is shown in Fig.\ \ref{fjhisto}. For larger values of $\eta$ the main features of
1014: $c(x_n,\eta)$ for the exponential and the Gaussian case as described in
1015: Sec.\ \ref{Gaussian} and \ref{symexpo} are reproducable. For larger values of $\eta$
1016: we find that if $-\sigma \eta < a_{n,k} -\sigma \eta/2 $ $c(a_n,\eta)$ is either
1017: larger than zero in the asymptotically Gaussian case ($k=144$) or equal to zero in the
1018: asymptotically exponential case ($k=10$).}
1019: \end{figure*}
1020: %%%%%%%%%%%%%%%%%%
1021: Thus the incremental data sets $a_{n,k}$ provides us with the opportunity to test the
1022: results for statistical predictions within Gaussian and exponential
1023: distributed i.i.d.\ random numbers on a data set, which exhibits
1024: correlated structures.
1025:
1026: We are now interested in predicting increments of the acceleration
1027: $a_{n+j,k} - a_{n,k} \geq \eta$ in the incremental data sets $a_{n,k}=v_{n+k}
1028: -v_n$.
1029: %
1030: In the following we concentrate on the incremental data set $a_{n,10}$, which
1031: has an asymptotically exponential PDF and the data set $a_{n,144}$, which has
1032: an asymptotically Gaussian PDF.
1033: %
1034: Furthermore we focus on increments between
1035: relatively large time steps, i.e., $j=285$, so that the short-range persistence of the
1036: process does not prevent large events from occuring.
1037: %
1038: As in the previous
1039: sections we are hence exploiting the statistical properties of the time series
1040: to make predictions, rather than the dynamical properties.
1041:
1042: We can now use the evaluation algorithm which was tested on the previous
1043: examples to evaluate the condition for these data sets.
1044: %
1045: The results are shown in Fig.\ \ref{fjcondi}.
1046: %
1047: We find that at least for larger values of $\eta$ the main features of
1048: $c(x_n,\eta)$ for the exponential and the Gaussian case as described in
1049: Sec. \ref{Gaussian} and \ref{symexpo} are also present in the free jet
1050: data.
1051: %
1052: For larger values of $\eta$, $c(a_{n,k},\eta)$ is either
1053: larger than zero in the Gaussian case ($k=144$) or equal to zero in the
1054: exponential case ($k=10$) in the region of interesting precursory variables, i.e., small
1055: values of $a_{n,k}$.
1056: %%%%%%%%%%
1057: \begin{figure*}
1058: \centerline{
1059: \hspace{2cm}
1060: \epsfig{file=figure13.eps,width=6cm, angle=-90}
1061: \epsfig{file=figure14.eps,width=6cm, angle= -90}
1062: }
1063: \caption{\label{fjrocs}Transition from exponential ROC curves (a) to Gaussian
1064: ROC curves (b). In the exponential case ($k=10$),
1065: shown in (a) the ROC curves for different event magnitude
1066: $\eta$ are almost the same, although the range of $\eta$ is larger ($\eta \in
1067: (0,6.71)$) than in the Gaussian case shown in (b). For $k=144$ the ROC curves are further
1068: apart, which corresponds to the results for Gaussian ROC curves (see
1069: Sec.\ \ref{Gaussian}) }
1070: \end{figure*}
1071: %%%%%%%%%%%%%%
1072:
1073: However, the presence of the exponential and the Gaussian distributions is more
1074: prominent in the corresponding ROC curves. For the free jet data set, the
1075: predictions were made with an algorithm similar to the one described in
1076: Sec. \ref{pre}. Instead of a specific precursory
1077: structure, which corresponds to the maximum of the likelihood, we use here a
1078: threshold of the likelihood as a precursor. In this setting we give an alarm
1079: for an extreme event, whenever the likelihood that an extreme event follows an
1080: observation is larger than a given threshold value.
1081:
1082: In the exponential case ($k=10$)
1083: shown in Fig.\ \ref{fjrocs}(a) the ROC curves for different event magnitude
1084: $\eta$ almost coincide, although the range of $\eta$ is larger ($\eta \in
1085: (0,6.71)$) than in the Gaussian case shown in Fig.\ \ref{fjrocs} (b). For $k=144$ the ROC curves are further
1086: apart, which corresponds to the results of Secs. \ref{Gaussian} and \ref{symexpo}.
1087:
1088: This example of the free jet data set shows that the specific dependence of
1089: the ROC curve on the event magnitude can also in the case of correlated data
1090: sets be characterized by the PDF of the underlying process.
1091: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
1092: %\newpage
1093: \section{Conclusions \label{conclusions}}
1094: We study the magnitude dependence of the quality of predictions for increments in a
1095: time series which consists in sequences of i.i.d.\ random numbers and in
1096: acceleration increments measured in a free jet flow.
1097: %
1098: Using the first part of
1099: the increment $x_n$ as a
1100: precursory variable we predict large increments $x_{n+1} -x_n$ via statistical
1101: considerations.
1102: %
1103: In order to measure the quality of the predictions we use ROC curves.
1104: %
1105: Furthermore we introduce a quantitative criterion which can determine
1106: whether larger or smaller events are better predictable. This criterion is
1107: tested for time series of Gaussian, exponential and Pareto i.i.d.\ random
1108: variables and for the increments of the acceleration in the free jet flow. The
1109: results obtained from the criterion comply nicely with the corresponding ROC-curves.
1110: Note that for both, the numerical evaluation of the condition and the
1111: ROC-plots, we used only event magnitudes $\eta$ for which we found at least $1000$
1112: events, so that the observed effects are not due to a lack of statistics of
1113: the large events.
1114:
1115: In the sequence of Gaussian i.i.d.\ random numbers, we find that large
1116: increments are better predictable the larger they are. In the Pareto
1117: distributed time series we observe that in slowly decaying power laws larger
1118: events are harder to predict, the larger they are. We find no significant
1119: dependence on the event-magnitude for the sequence of exponentially i.i.d.\ random numbers.
1120:
1121: While the condition can be easily evaluated analytically, it is not that easy
1122: to compute numerically from observed data, since the
1123: calculation implies evaluating the derivatives of numerically obtained
1124: distributions.
1125: %
1126: Using Savitzky-Golay filters improved the results, but
1127: especially in the limit of larger events, where the distributions are
1128: difficult to sample, one cannot trust the results of the numerically
1129: evaluated criterion.
1130: %
1131: However, it is still possible to apply the criterion by
1132: fitting a PDF to the distribution of the underlying process and then evaluate
1133: the criterion analytically.
1134:
1135: Although the magnitude dependence of the quality of predictions was observed in
1136: different contexts and for different measures of predictability, in this
1137: contribution only ROC curves were used. In order to exclude the possibility
1138: that the effect is specific to the ROC curve, future works should also include
1139: other measures of predictability.
1140:
1141: Reviewing the results for the Gaussian case and the slowly decaying power law
1142: from a philosophic point of view one can conclude that nature allows us to
1143: predict large events from the most frequently occuring distribution
1144: easily. However in Gaussian distributions very large events are rare and
1145: therefore less likely to cause damage. Whereas in the less frequently occurring
1146: distributions with heavy power-law tails, large events are especially hard to predict.
1147: %
1148: Therefore one can assume, that rare large impact events of processes with power-law
1149: distributions will remain unpredictable, although their prediction would be highly desirable.
1150:
1151: \acknowledgements
1152: We thank J. Peinke and his group for supplying us with the free jet data.
1153: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
1154: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
1155: \begin{appendix}
1156: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
1157:
1158: \section{Obtaining the analytic expression for the likelihood, the joint and the aposterior PDFs for increments in stochastical processes \label{appA}}
1159: An analytic expression for a filter which selects the PDF of our
1160: extreme increments $x_{n+1} - x_n \geq d$ out of the PDFs of the underlying stochastic process can be
1161: obtained through the Heaviside function $ \Theta( x_{n+1} - x_{n}-d)$.
1162: %
1163: (Note that $d$ is not scaled by the
1164: standard deviation, i.e., $d =\sigma\eta$.)
1165: %
1166: This filter is then applied to the joint PDF $j(x_0, x_1, ...,x_{n-k+1}, x_{n-k+2},
1167: ..., x_{n})$ of a stochastic process or to be more precise to the likelihood
1168: $L(x_{n+1}|x_0, x_1, ...,x_{n-k+1}, x_{n-k+2}, ..., x_{n})$ that the $n+1$
1169: step follows the previously obtained values. If we condition only on the last
1170: $k$ values, we neglect the dependence on the past.
1171: %
1172: The likelihood that an event Y(d)=1 follows in the $n+1$th step can then be
1173: obtained by multiplication with $\Theta( x_{n+1} - x_{n}-d)$.
1174: \begin{widetext}
1175: \begin{eqnarray}
1176: L(Y_{n+1}(d)=1| \mathbf{x}_{(n-k+1,n)}) & = & \Theta( x_{n+k} - x_{n}-d) L(x_{n+1}|\mathbf{x}_{(k,n)} ),
1177: \end{eqnarray}
1178:
1179: where $\mathbf{x}_{(n-k+1,n)}= (x_{n-k+1},x_{n-k+2}, ...,x_n) $ as defined in Sec.\ \ref{pre}.
1180: \end{widetext}
1181: If the resulting
1182: expression is nonzero, the condition of the extreme event in Eq.\ (\ref{e0}) is
1183: fulfilled and for $x_{n+1}$ and $x_{n}$ the following relation holds:
1184: %
1185: \begin{eqnarray}
1186: x_{n+1} & = & x_{n} + d + \gamma \label{gammadef} \quad (\gamma \in \mathbb R,
1187: \gamma \geq 0) \quad. \label{gamma0}
1188: \end{eqnarray}
1189: %
1190: Hence it is possible to express the likelihood in terms of $x_{n}$, which is a part of the precursory structure. We can use the integral representation of the Heaviside function with appropriate substitutions to obtain
1191: %
1192: \begin{widetext}
1193: \begin{eqnarray}
1194: L(Y_{n+1}(d)=1|\mathbf{x}_{(n-k+1,n)}) &=& \int_{0}^{\infty} L(x_{n} + d + \gamma|\mathbf{x}_{(n-k+1,n)} )\;d\gamma.\quad\label{int1}
1195: \end{eqnarray}
1196: \end{widetext}
1197: %
1198:
1199: Hence the joint PDF, the aposterior PDF and the total probability to find
1200: increments are given by
1201: \begin{widetext}
1202: \begin{eqnarray}
1203: j\bigl(\mathbf{x}_{(n-k+1,n)},Y_{n+1}(d)=1\bigr) & = & j\bigl(\mathbf{x}_{(n-k+1,n)}\bigr) \cdot L\bigl(Y_{n+1}(d)=1|\mathbf{x}_{(n-k+1,n)}\bigr)\\
1204: \rho\bigl(\mathbf{x}_{(n-k+1,n)}|Y_{n+1}(d)=1\bigr) & = & \frac{j\bigl(\mathbf{x}_{(n-k+1,n)},Y_{n+1}(d)=1\bigr)}{P(Y(d)=1)},\\
1205: P(Y(d)=1) &=& \int_{-\infty}^{\infty} dx_{n-k+1} \int_{-\infty}^{\infty} dx_{n-k+2} ... \int_{-\infty}^{\infty} dx_n \; j(\mathbf{x}_{(0,n-k)}\mathbf{x}_{(n-k+1,n)},Y_{n+1}(d)=1). \label{appendixptotal}
1206: \end{eqnarray}
1207: \end{widetext}
1208:
1209: Whether we can acess a given stochastical process analytically or not depends on the question of whether the integrals in Eq.\ (\ref{appendixptotal}) can be solved or not.
1210:
1211: If we are interested in the prediction of threshold crossings instead of
1212: increments, we can interpret $\eta$ as the magnitude of the threshold and set
1213: $x_n=0$ in order to obtain the corresponding expressions for the likelihood,
1214: the joint PDF, the aposterior PDF, and the total probability.
1215: \end{appendix}
1216: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
1217: \begin{thebibliography}{99}
1218: %1
1219: \bibitem{Jackson} David D. Jackson, {\sl Hypothesis testing and earthquake
1220: prediction}, Proc. Natl. Acad. Sci. U.S.A.\ {\bf 93} 3772 (1996).
1221: %3
1222: \bibitem{Physa} H. Kantz, D. Holstein, M. Ragwitz and N. K. Vitanov, {\sl Markov
1223: chain model for turbulent wind speed data}, Physica {\bf A 342} 315 (2004).
1224: %10
1225: \bibitem{Euromech}
1226: Holger Kantz, Detlef Holstein, Mario Ragwitz and Nikolay K. Vitanov, {Short
1227: time prediction of wind speeds from local measurements}, in: {\sl Wind Energy
1228: -- Proceedings of the EUROMECH Colloquium}, edited by J. Peinke, P. Schaumann and S. Barth, (Springer, New York, 2006).
1229: %
1230: \bibitem{Sarah2} S.\ Hallerberg , E.\ G.\ Altmann, D.\ Holstein and H.\ Kantz, {\sl Precursors of Extreme
1231: Increments}, Phys.\ Rev.\ E {\bf 75}, 016706 (2007)
1232: %
1233: \bibitem{Fatehme} M.Reza Rahimi Tabar, M. Sahimi, F. Ghasemi, K. Kaviani,
1234: M. Allamehzadeh, J. Peinke, M. Mokhtaru and M. Vesaghi,
1235: M. D. Niry, A. Bahraminasab, S. Tabatabai, S. Fayazbakhsh and M. Akbari {\sl
1236: Short-Term Prediction of Medium- and Large-Size Earthquakes Based on Markov
1237: and Extended Self-Similarity Analysis of Seismic Data} arXiv:physics/0510043 v1 6 Oct 2005.
1238: %
1239: \bibitem{Sandpile} A. B. Shapoval and M. G. Shrirman, {\sl How size of target
1240: avalanches influence prediction efficiency}, International Journal of Modern
1241: Physics C, {\bf 17}, 1777 (2006).
1242: %
1243: \bibitem{Johnson1} D. Lamper, S.D. Howison and N.F. Johnson, {\sl Predictability
1244: of Large Future Changes in a Competitive Evolving Population},
1245: Phys. Rev. Lett. {\bf 88}, 017902 (2002).
1246: %
1247: \bibitem{Egans} J. P. Egan, {\sc Signal detection theory and ROC analysis},
1248: Academic Press, New York 1975.
1249: %17
1250: \bibitem{Box-Jen} G. E. P. Box, G. M. Jenkins and G. C. Reinsel, {\sc Time Series
1251: Analysis} (Prentice-Hall, Inc., Eaglewood Clif., NJ, 1994).
1252: %8
1253: \bibitem{Brockwell} P.J. Brockwell and R.A. Davis, {\sc Time Series: Theory and
1254: Methods} (Springer, New York, 1998).
1255: %9
1256: \bibitem{likelihood} J. M. Bernado and A. F. M. Smith, {\sc Bayesian Theory} (Wiley, New
1257: York, 1994).
1258: %15
1259: \bibitem{Swets1}D. M. Green and J. A. Swets, {\sl Signal detection theory and
1260: psychophysics.} (Wiley, New York, 1966).
1261: %16
1262: \bibitem{Pepe} M. S. Pepe, {\sc The Statistical Evaluation of Medical Tests for
1263: Classification and Prediction}, (Oxford University Press, New York, 2003).
1264: %
1265: \bibitem{bandi} J. Broecker and L. A. Smith {\sl Scoring Probabilistic
1266: Forecasts: On the Importance of Being Proper}, Weather and Forecasting {\bf 22}, 382 (2007).
1267: %19
1268: \bibitem {Sornette2} A. Johansen and D. Sornette,
1269: {\sl Stock market crashes are outliers}, European Physical Journal B {\bf 1}, 141 (1998).
1270: %5
1271: \bibitem{stocks} N. Vandewalle, M. Ausloos, P. Boveroux, et al.,
1272: {\sl How the financial crash of October 1997 could have been predicted},
1273: European Physical Journal B {\bf 4} 139 (1998).
1274: %6
1275: \bibitem{Savgol} A. Savitzky and M. J. E. Golay {\sl Smoothing and
1276: Differentiation of Data by Simplified Least Squares Procedures}, Analytical
1277: Chemistry {\bf 36} 1627 (1964).
1278: %
1279: \bibitem{numrecipes} W. H. Press, {\sc Numerical recipes in C} (Cambridge
1280: University Press, Cambridge, England 1992).
1281: %
1282: \bibitem{Abram} M. Abramowitz, and I. A. Stegun, {\sc Handbook of Mathematical
1283: Functions}, (Dover, New York, 1972).
1284: %
1285: \bibitem{Feller} W. Feller, {\sc An Introduction to Probability Theory and Its
1286: Applications} (Wiley, New York (1970), Vol. II.
1287: %
1288: \bibitem{Peinke} C. Renner, J. Peinke and R. Friedrich, {\sl Experimental
1289: indications for Markov properties of small-scale turbulence},
1290: J. Fluid. Mech. {\bf 433}, 383 (2001).
1291: %
1292: \bibitem{vanAtta} C. W. Van Atta and J. Park, { Statistical self-similarity
1293: and intertial subrange turbulence}, in {\sl Statistical Models and
1294: Turbulence}, edited by M. Rosenblatt and C. W. Van Atta, Lect. Notes in Phys., Vol. 12,
1295: (Springer, Berlin, 1972), pp. 402-426.
1296: %
1297: \bibitem{Frisch1990} Y. Gagne, E. Hopfinger and U. Frisch {A new universal
1298: scaling for fully developed turbulence: the distribution of velocity
1299: increments} in {\sl New Trends in Nonlinear Dynamics and Pattern-Forming
1300: Phenomena} edited by P. Coullet and P. Huerre, NATO ASI (Plenum Press, New York 1990),
1301: Vol. 237, pp. 315-319.
1302: \bibitem{Frisch1995} U. Frisch {\sc Turbulence} Cambridge University Press, (Cambridge, England, 1995).
1303: %
1304: \end{thebibliography}
1305: %
1306: \end{document}
1307:
1308: