1: \documentclass[a4paper,12pt]{article}%
2: \usepackage{amsmath}
3: \usepackage{amsfonts}
4: \usepackage{amssymb}
5: \usepackage[dvips]{graphicx}%
6: \setcounter{MaxMatrixCols}{30}
7: %TCIDATA{OutputFilter=latex2.dll}
8: %TCIDATA{Version=5.00.0.2570}
9: %TCIDATA{CSTFile=40 LaTeX article.cst}
10: %TCIDATA{Created=Saturday, February 04, 2006 16:47:20}
11: %TCIDATA{LastRevised=Monday, October 02, 2006 09:44:39}
12: %TCIDATA{<META NAME="GraphicsSave" CONTENT="32">}
13: %TCIDATA{<META NAME="SaveForMode" CONTENT="1">}
14: %TCIDATA{<META NAME="DocumentShell" CONTENT="Standard LaTeX\Blank - Standard LaTeX Article">}
15: \newtheorem{theorem}{Theorem}
16: \newtheorem{acknowledgement}[theorem]{Acknowledgement}
17: \newtheorem{algorithm}[theorem]{Algorithm}
18: \newtheorem{axiom}[theorem]{Axiom}
19: \newtheorem{case}[theorem]{Case}
20: \newtheorem{claim}[theorem]{Claim}
21: \newtheorem{conclusion}[theorem]{Conclusion}
22: \newtheorem{condition}[theorem]{Condition}
23: \newtheorem{conjecture}[theorem]{Conjecture}
24: \newtheorem{corollary}[theorem]{Corollary}
25: \newtheorem{criterion}[theorem]{Criterion}
26: \newtheorem{definition}[theorem]{Definition}
27: \newtheorem{example}[theorem]{Example}
28: \newtheorem{exercise}[theorem]{Exercise}
29: \newtheorem{lemma}[theorem]{Lemma}
30: \newtheorem{notation}[theorem]{Notation}
31: \newtheorem{problem}[theorem]{Problem}
32: \newtheorem{proposition}[theorem]{Proposition}
33: \newtheorem{remark}[theorem]{Remark}
34: \newtheorem{solution}[theorem]{Solution}
35: \newtheorem{summary}[theorem]{Summary}
36: \newenvironment{proof}[1][Proof]{\noindent\textbf{#1.} }{\ \rule{0.5em}{0.5em}}
37: \begin{document}
38:
39: \author{Andr\'{e} MAS$^{\ast}$, Besnik PUMO$^{\dag}\medskip$\\$\left( ^{\ast}\right) $ {\small Institut de Mod\'{e}lisation
40: Math\'{e}matique de Montpellier,}\\{\small cc 051, Place Eug\`{e}ne Bataillon,}\\{\small 34095, Montpellier Cedex 5, France,}\\{\small mas@math.univ-montp2.fr\medskip\ }\\$\left( ^{\dag}\right) $ {\small Unit\'{e} de Statistiques,}\\{\small UMR A 462 SAGAH,}\\{\small INH, Angers, France}}
41: \title{Functional linear regression with derivatives}
42: \date{}
43: \maketitle
44:
45: \begin{abstract}
46: We introduce a new model of linear regression for random functional inputs
47: taking into account the first order derivative of the data. We propose an
48: estimation method which comes down to solving a special linear inverse
49: problem. Our procedure tackles the problem through a double and synchronized
50: penalization. An asymptotic expansion of the mean square prevision error is
51: given. The model and the method are applied to a benchmark dataset of
52: spectrometric curves and compared with other functional models.
53:
54: \end{abstract}
55:
56: \textbf{Keywords} : Functional data, Linear regression model, Differential
57: operator, Penalization, Spectrometric curves.\bigskip
58:
59: \section{Introduction}
60:
61: Functional Data Analysis is a well-known area of modern statistics. Advances
62: in computer sciences make it now possible to collect data from an underlying
63: continuous-time processe, say $\left( \xi_{t}\right) _{t\geq0}$, at high
64: frequencies. The traditional point of view consisting in discretizing $\left(
65: \xi_{t}\right) $ at $t_{1},...,t_{p}$ and studying it by classical
66: multidimensional tools is outperformed by interpolation methods (such as
67: splines or wavelets). These techniques provide the statistician with a
68: reconstructed curve on which inference may be carried out through what we may
69: call "functional models" i.e. versions of the classical multidimensional
70: models designed and suited for data that are curves. Thus, functional PCA,
71: ANOVA or Canonical Analysis -even density estimation for curves or processes
72: have been investigated. We refer to Ramsay, Silverman (1997, 2002), Bosq
73: (2000), Ferraty Vieu (2006) for monographs on functional data analysis.
74: Recently many authors focused on various versions of the regression model
75: introduced by Ramsay and Dalzell (1991) :%
76: \begin{equation}
77: y_{i}=\int_{0}^{T}X_{i}\left( t\right) \rho\left( t\right) dt+\varepsilon
78: _{i} \label{mod1}%
79: \end{equation}
80: where we assume that the sample $\left( \left( y_{1,}X_{1}\right)
81: ,...,\left( y_{n},X_{n}\right) \right) $ is made of independent copies from
82: $\left( y_{,}X\right) .$ Each $X_{i}=\left( X_{i}\left( t\right) \right)
83: _{t\in\left[ 0,T\right] }$ is a curve defined on the set $\left[
84: 0,T\right] ,$ $T>0$, $y_{i}$ is a real number, $\varepsilon_{i}$ is a white
85: noise and $\rho$ is an unknown function to be estimated. In other words the
86: $X_{i}$'s are random elements defined on an abstract probability space and
87: taking values in a function space, say $\mathcal{F}.$ The vector space
88: $\mathcal{F}$ endowed with norm $\left\Vert \cdot\right\Vert _{\mathcal{F}}$
89: will be described soon.We refer for instance to Cardot, Mas, Sarda (2006) or
90: Cai, Hall (2006) for recent results.
91:
92: In this article we study a new (linear) regression model defined below derived
93: from (\ref{mod1}) and echoing the recent paper of Mas and Pumo (2006). The key
94: idea relies on the fact that most statisticians dealing with functional data
95: do not fully enjoy their functional properties. For instance in several models
96: integrals such as%
97: \[
98: \int X_{i}\left( s\right) X_{j}\left( s\right) ds
99: \]
100: are computed. The integral above is nothing but a scalar product. Nevertheless
101: derivatives were not given the same interest. Explicit calculations of
102: derivatives sometimes appear indirectly in kernel methods (when estimating the
103: derivatives of the density or the regression function) or through seminorms or
104: norms on $\mathcal{F}$. But surprisingly $X_{i}^{\prime}$ (or $X_{i}^{\left(
105: m\right) }$) never appear in the models themselves whereas people dealing
106: with functional data often say that "derivatives contain much information,
107: sometimes more than the initial curves themselves". Our starting idea is the
108: following. Since in a functional data framework, the curve-data are
109: explicitely known and not just discretized, their derivatives may also be
110: explicitely computed. As a consequence these derivatives may be "injected" in
111: the model, which may enhance its prediction power. The reader is referred to
112: the forthcoming display (\ref{modele}) for an immediate illustration and to
113: Mas, Pumo (2006) for a first article dealing with a functional autoregressive
114: model including derivatives.
115:
116: The paper is rather theoretic even if it is illustrated by a real case study.
117: It is organized as follows. The next section provides the mathematical
118: material, dealing with Hilbert spaces and linear operators, then the model is
119: introduced. The next section is devoted to presenting the estimation method
120: and its stumbling stones. The main results are given before we focus on a real
121: case application to food industry. The last section contains the derivation of
122: the theorems.
123:
124: \section{About Hilbert spaces and linear operators}
125:
126: Silverman (1996) provided a theoretical framework for a smoothed PCA. Jim
127: Ramsay (2000) enlightened the very wide scope of differential equations in
128: statistical modelling. Our work is in a way based on this mathematically
129: involved article. We are aiming at proving that derivatives may be handled in
130: statistical models quite easily when the space $\mathcal{F}$ is well-chosen.
131:
132: The choice of the space $\mathcal{F}$ is crucial. We have to think that if
133: $X\in\mathcal{F}$, $X^{\prime}$ does not necessarily belong to $\mathcal{F}$
134: but to another space $\mathcal{F}^{\prime}$ that may be tremendously different
135: (larger) than $\mathcal{F}$. We decide to take $\mathcal{F}=W^{2,1},$ the
136: Sobolev space of order $\left( 2,1\right) $ defined by
137: \[
138: W^{2,1}=\left\{ u\in L^{2}\left[ 0,1\right] ,u^{\prime}\in L^{2}\left[
139: 0,1\right] \right\}
140: \]
141: for at least three reasons :
142:
143: \begin{itemize}
144: \item If $X\in\mathcal{F}$, $X^{\prime}\in L^{2}\left[ 0,1\right] $ which is
145: a well known space.
146:
147: \item Both spaces are Hilbert spaces as well as%
148: \[
149: W^{2,p}=\left\{ u\in L^{2}\left[ 0,1\right] ,u^{\left( p\right) }\in
150: L^{2}\left[ 0,1\right] \right\} .
151: \]
152: This is of great interest for mathematical reasons : bases are denumerable,
153: projections operators are easy to handle, covariance operators admit spectral
154: representations, etc.
155:
156: \item The classical interpolation methods mentioned above (splines and
157: wavelets) provide estimates belonging to Sobolev spaces. So from a practical
158: point of view $W^{2,1}$ -and in general $W^{m,p},$ $\left( m,p\right)
159: \in\mathbb{N}^{2},$ (see Adams and Fournier (2003) for definitions)- is a
160: natural space in which our curves should be imbedded.
161: \end{itemize}
162:
163: In the sequel $W^{2,1}$ will be denoted $W$ and $W^{2,0}=L^{2}$ will be
164: denoted $L$ for the sake of simplicity. We keep in mind that $W$ (resp. $L$)
165: could be replaced by a space of higher smoothness index : $W^{2,p}$ where
166: $p>1$ (resp. $W^{2,p-1}$). The spaces $W$ and $L$ are separable Hilbert spaces
167: endowed with scalar product :%
168: \begin{align*}
169: \left\langle u,v\right\rangle _{W} & =\int_{0}^{1}u\left( t\right)
170: v\left( t\right) dt+\int_{0}^{1}u^{\prime}\left( t\right) v^{\prime
171: }\left( t\right) dt.\\
172: \left\langle u,v\right\rangle _{L} & =\int_{0}^{1}u\left( t\right)
173: v\left( t\right) dt
174: \end{align*}
175: and with associated norms $\left\Vert \cdot\right\Vert _{W}$ and $\left\Vert
176: \cdot\right\Vert _{L}$. We refer to Ziemer (1989) or to Adams and Fournier
177: (2003) for monographs dedicated to Sobolev spaces. Obviously if we set
178: $Du=u^{\prime}$ then $D$ maps $W$ onto $L$ ($D$ is the ordinary differential
179: operator)$.$ Furthermore Sobolev's imbedding theorem ensures that (see Adams
180: and Fournier (2003) Theorem 4.12 p.85) that%
181: \[
182: \left\Vert Du\right\Vert _{L}\leq C\left\Vert u\right\Vert _{W}%
183: \]
184: (where $C$ is some constant which does not depend on $u$) i.e. $D$\textbf{ }is
185: a bounded operator from\textbf{ }$W$\textbf{ }to\textbf{ }$L.$ This is a
186: crucial point to keep in mind and the fourth reason why the functional space
187: was chosen to be $W^{2,1}$ : the differential operator $D$ may be viewed as a
188: continuous linear mapping from $W$ to $L$.
189:
190: Within all the paper and especially all along the proofs we will need basic
191: notions about operator theory. We recall a few important facts. A linear
192: mapping $T$ from a Hilbert space $H$ to another Hilbert space $H^{\prime}$ is
193: continuous whenever%
194: \begin{equation}
195: \left\Vert T\right\Vert _{\infty}=\sup_{x\in H}\frac{\left\Vert Tx\right\Vert
196: _{H^{\prime}}}{\left\Vert x\right\Vert _{H}}<+\infty. \label{normsup}%
197: \end{equation}
198: The adjoint of operator $T$ will be classically denoted $T^{\ast}$. Some
199: finite rank operators are defined by means of the tensor product : if $u$ and
200: $v$ belong to $H$ and $H^{\prime}$ respectively $u\otimes_{H}v$ is the
201: operator defined on $H$ by, for all $h\in H$~:%
202: \[
203: \left( u\otimes_{H}v\right) \left( h\right) =\left\langle u,h\right\rangle
204: _{H}v.
205: \]
206:
207:
208: \textbf{Compact operators} : Amongst linear operators the class of compact
209: operators is one of the best known. Compact operators generalize matrix to the
210: infinite-dimensional setting and feature nice properties. The general
211: definition of compact operators may be found in Dunford Schwartz (1988) or
212: Gohberg, Goldberg and Kaashoek (1991) for instance. By $\mathcal{C}_{H}$
213: (resp. $\mathcal{C}_{HH^{\prime}}$) we denote the space of compact operators
214: on the Hilbert space $H$ (resp. mapping the Hilbert space $H$ onto $H^{\prime
215: }$). If $T$ is a compact operator from a Hilbert space $H_{1}$ to another
216: Hilbert space $H_{2},$ $T$ admits the Schmidt decomposition :%
217: \begin{equation}
218: T=\sum_{k\in\mathbb{N}}\mu_{k}\left( u_{k}\otimes v_{k}\right)
219: \label{Decomp.Schmidt}%
220: \end{equation}
221: where $u_{k}$ (resp. $v_{k}$) is a complete orthonormal system in $H_{1}$
222: (resp. in $H_{2}$) and $\mu_{k}$ are the characteristic numbers of $T$ (i.e.
223: the square root of the eigenvalues of $T^{\ast}T$) and
224: \[
225: \lim_{k\rightarrow+\infty}\mu_{k}=0.
226: \]
227: From (\ref{normsup}) we obtain%
228: \[
229: \left\Vert T\right\Vert _{\infty}=\sup_{k}\left\{ \mu_{k}\right\} .
230: \]
231: When $T$ is symmetric $\mu_{k}$ is the $k^{th}$ eigenvalue of $T$ (then
232: $u_{k}=v_{k}$). In this situation and from (\ref{Decomp.Schmidt}) one may
233: define the square root of $T$ whenever $T$ maps $H$ ont $H$ and is positive :
234: $T^{1/2}$ is still a linear operator defined by :%
235: \begin{equation}
236: T^{1/2}=\sum_{k\in\mathbb{N}}\sqrt{\mu_{k}}\left( u_{k}\otimes u_{k}\right)
237: . \label{racop}%
238: \end{equation}
239: Note that finite rank operators are always compact.\newline%
240: \textbf{Hilbert-Schmidt operators} : We also mention the celebrated space of
241: Hilbert-Schmidt operators $\mathcal{HS}\left( H_{1},H_{2}\right) $ - a
242: subspace of $\mathcal{C}_{H_{1}H_{2}}.$ Let $\left( u_{i}\right) _{i\geq0}$
243: be a basis of $H_{1}$ then $T\in\mathcal{HS}\left( H_{1},H_{2}\right) $
244: whenever%
245: \[
246: \sum_{i=1}^{+\infty}\left\Vert T\left( u_{i}\right) \right\Vert _{H_{2}}%
247: ^{2}<+\infty.
248: \]
249: The space $\mathcal{HS}$ is itself a separable Hilbert space endowed with
250: scalar product%
251: \[
252: \left\langle T,S\right\rangle _{\mathcal{HS}}=\sum_{i=1}^{+\infty}\left\langle
253: T\left( u_{i}\right) ,S\left( u_{i}\right) \right\rangle _{H_{2}}%
254: \]
255: and $\left\langle T,S\right\rangle _{\mathcal{HS}}$ does not depend on the
256: choice of the basis $\left( u_{i}\right) _{i\geq0}.$ Finally the following
257: bound is valid for all $T\in\mathcal{HS}$ :%
258: \[
259: \left\Vert T\right\Vert _{\infty}\leq\left\Vert T\right\Vert _{\mathcal{HS}}.
260: \]
261: \newline\textbf{Unbounded operators} : If $T$ is a one to one (injective)
262: selfadjoint compact operator mapping a Hilbert space $H$ onto $H$, $T$ admits
263: an inverse $T^{-1}$. The operator $T^{-1}$ is defined on a dense (and
264: distinct) subspace of $H$ :%
265: \[
266: \mathcal{D}\left( T^{-1}\right) =\left\{ x=\sum_{p\in\mathbb{N}}x_{p}%
267: u_{p}:\sum_{p\in\mathbb{N}}\dfrac{x_{p}^{2}}{\mu_{p}^{2}}<+\infty\right\} .
268: \]
269: It is unbounded which also means that $T^{-1}$ is continuous at no point for
270: which it is defined and $\left\Vert T^{-1}\right\Vert _{\infty}=+\infty$.
271:
272: \section{The model}
273:
274: We are now in position to introduce this (random input - linear) regression
275: model :%
276:
277: \begin{equation}
278: y_{i}=\left\langle \phi,X_{i}\right\rangle _{W}+\left\langle \psi
279: ,X_{i}^{\prime}\right\rangle _{L}+\varepsilon_{i} \label{modele}%
280: \end{equation}
281: where all random variables are assumed to be centered. The main result of the
282: paper (see next section) gives an asymptotic expansion for the mean square
283: prediction error in (\ref{modele}).
284:
285: The unknown functions $\phi$ and $\psi$ belong to $W$ and $L$ respectively.
286:
287: Obviously we are going to face two issues :
288:
289: \begin{itemize}
290: \item Studying the identifiability of $\phi$ and $\psi$ in the model above.
291:
292: \item Providing a consistent estimation procedure for $\phi$ and $\psi$.
293: \end{itemize}
294:
295: From now on we suppose that :%
296: \[
297: \mathbf{A1}:\left\Vert X\right\Vert _{W}<M\quad a.s.
298: \]
299:
300:
301: This assumption could be relaxed for milder moment assumptions. We claim that
302: our main result holds whenever%
303: \[
304: \mathbf{A}^{\prime}\mathbf{1}:\mathbb{E}\left\Vert X\right\Vert _{W}^{8}<M.
305: \]
306: is true. But considering $\mathbf{A}^{\prime}\mathbf{1}$ would lead us to
307: longer and more intricate methods of proof.
308:
309: \section{Estimation procedure\label{section.estimation.procedure}}
310:
311: \subsection{The moment method}
312:
313: Inference is based on moment formulas. From (\ref{modele}) we derive the two
314: following normal equation -multiply with $\left\langle X_{i},\cdot
315: \right\rangle $ and $\left\langle X_{i}^{\prime},\cdot\right\rangle $
316: successively then take expectation :%
317: \begin{equation}
318: \left\{
319: \begin{tabular}
320: [c]{l}%
321: $\delta=\Gamma\phi+\Gamma^{\prime}\psi,$\\
322: $\delta^{\prime}=\Gamma^{\prime\ast}\phi+\Gamma^{\prime\prime}\psi.$%
323: \end{tabular}
324: \ \ \ \ \ \ \ \ \ \ \right. \label{syst}%
325: \end{equation}
326: where $\Gamma,$ $\Gamma^{\prime},$ $\Gamma^{\prime\ast},$ $\Gamma
327: ^{\prime\prime}$ are the covariance and cross-covariance of the couple
328: $\left( X_{i},X_{i}^{\prime}\right) _{1\leq i\leq n}$ defined by :%
329: \begin{align*}
330: \Gamma & =\mathbb{E}\left( X\otimes_{W}X\right) ,\;\Gamma^{\prime\ast
331: }=\mathbb{E}\left( X\otimes_{W}X^{\prime}\right) ,\\
332: \Gamma^{\prime} & =\mathbb{E}\left( X^{\prime}\otimes_{L}X\right)
333: ,\;\Gamma^{\prime\prime}=\mathbb{E}\left( X^{\prime}\otimes_{L}X^{\prime
334: }\right) ,
335: \end{align*}
336: and%
337: \[
338: \delta=\mathbb{E}\left( yX\right) \in W,\quad\delta^{\prime}=\mathbb{E}%
339: \left( yX^{\prime}\right) \in L.
340: \]
341:
342:
343: Under assumption $\mathbf{A1}$ or $\mathbf{A}^{\prime}\mathbf{1}$ the
344: covariance operators belong to $\mathcal{HS}\left( W\right) $,
345: $\mathcal{HS}\left( W,L\right) $, $\mathcal{HS}\left( L,W\right) $ or to
346: $\mathcal{HS}\left( L\right) $. Besides the covariance and cross-covariance
347: mentioned above are linked through the relation%
348: \[
349: \Gamma^{\prime\ast}=D\Gamma,\quad\Gamma^{\prime\prime}=D\Gamma^{\prime}.
350: \]
351:
352:
353: Resolving the system (\ref{syst}) is apparently easy but we should be aware of
354: two facts :
355:
356: \begin{itemize}
357: \item Operators (here, $\Gamma,\Gamma^{\prime}..$.) do not commute !
358:
359: \item The inverse operators of $\Gamma,$ and $\Gamma^{\prime\prime}$ do not
360: necessarily exist and when they do, they are unbounded, i.e. not continuous
361: (recall that $\Gamma,$ and $\Gamma^{\prime\prime}$ are compact operators and
362: that compact operators have no bounded inverses).
363: \end{itemize}
364:
365: Before trying to solve (\ref{syst}) we will first study identifiability of the
366: unknown infinite dimensional parameter $\left( \phi,\psi\right) \in W\times
367: L$ in the next subsection. We complete our definitions and notations first.
368:
369: We start from a sample $\left( X_{i},X_{i}^{\prime}\right) _{1\leq i\leq n}%
370: $. By $\Gamma_{n},\Gamma_{n}^{\prime},\Gamma_{n}^{\prime\ast},\Gamma
371: _{n}^{\prime\prime},\delta_{n}$ and $\delta_{n}^{\prime}$ we denote the
372: empirical counterparts of the operators and vectors introduced above and based
373: on the sample $\left( y_{i},X_{i},X_{i}^{\prime}\right) _{1\leq i\leq n}$.
374: For example :%
375: \begin{align}
376: \Gamma_{n} & =\dfrac{1}{n}\sum_{k=1}^{n}X_{k}\otimes_{W}X_{k},\label{esti}\\
377: \Gamma_{n}^{\prime} & =\dfrac{1}{n}\sum_{k=1}^{n}X_{k}^{\prime}\otimes
378: _{L}X_{k},\nonumber\\
379: \delta_{n} & =\dfrac{1}{n-1}\sum_{k=1}^{n-1}y_{k}X_{k}.\nonumber
380: \end{align}
381:
382:
383: \subsection{Identifiability}
384:
385: Both equations in (\ref{syst}) are the starting point of the estimation
386: procedure. We should make sure that solutions to these equations are well and
387: uniquely defined. Suppose for instance that \textrm{Ker}$\Gamma\neq\left\{
388: 0\right\} $ and take $h$ in it. Now set $\widetilde{\phi}=\phi+h.$ Then
389: \[
390: \Gamma\widetilde{\phi}=\Gamma\phi+\Gamma h=\Gamma\phi.
391: \]
392: So $\Gamma\widetilde{\phi}=\Gamma\phi$ and since $\Gamma^{\prime\ast}=D\Gamma$
393: it is plain that $\Gamma^{\prime\ast}\widetilde{\phi}=\Gamma^{\prime\ast}%
394: \phi.$ Consequently $\widetilde{\phi}$ is another solution to (\ref{syst}).
395: There are indeed even infinitely many solutions in the space $\phi
396: +$\textrm{Ker}$\Gamma$. For similar reasons about $\psi$ we should impose
397: $\mathrm{Ker}T=\left\{ 0\right\} $ for $T=\left\{ \Gamma,\Gamma^{\prime
398: },\Gamma^{\prime\ast},\Gamma^{\prime\prime}\right\} .$ It turns out that the
399: only necessary assumption is%
400: \[
401: \mathbf{A2}:\mathrm{Ker}\Gamma=\mathrm{Ker}\Gamma^{\prime\prime}=\left\{
402: 0\right\} \mathrm{.}%
403: \]
404: It is easily seen that $\mathbf{A2}$ implies $\mathrm{Ker}\Gamma^{\prime
405: }=\mathrm{Ker}\Gamma^{\prime\ast}=\left\{ 0\right\} .$ With other words we
406: suppose that both operators $\Gamma$ and $\Gamma^{\prime\prime}$ above are one
407: to one.
408:
409: We are now ready to solve the identification problem.
410:
411: \begin{proposition}
412: \label{Ident}The couple $\left( \phi,\psi\right) \in W\times L$ is
413: identifiable for the moment method proposed in (\ref{syst}) if and only if
414: $\mathbf{A2}$ holds and $\left( \phi,\psi\right) \notin\mathcal{N}$ where
415: $\mathcal{N}$ is the vector subspace of $W\times L$ defined by :%
416: \begin{equation}
417: \mathcal{N=}\left\{ \left( \phi,\psi\right) :\phi+D^{\ast}\psi=0\right\} .
418: \label{fluke}%
419: \end{equation}
420:
421: \end{proposition}
422:
423: The above Proposition is slightly abstract but (\ref{fluke}) may be simply
424: rewritten: $\left( \phi,\psi\right) \in\mathcal{N}$ whenever for all
425: function $f$ in $W,$%
426: \[
427: \int\left( f\phi+f^{\prime}\phi^{\prime}+f^{\prime}\phi\right) =0
428: \]
429:
430:
431: Note that $\mathcal{N}$ is a closed set in $W\times L$. From now on we will
432: assume that :%
433: \[
434: \mathbf{A3}:\left( \phi,\psi\right) \notin\mathcal{N}\text{.}%
435: \]
436:
437:
438: \section{Definition of the estimates}
439:
440: The estimates stem from (\ref{syst}) which is a non invertible system. Under
441: assumption $\mathbf{A2}$ the solution exists and is unique :%
442: \begin{equation}
443: \left\{
444: \begin{tabular}
445: [c]{l}%
446: $\phi=\left( \Gamma-\Gamma^{\prime}\Gamma^{\prime\prime-1}\Gamma^{\prime\ast
447: }\right) ^{-1}\left[ \delta-\Gamma^{\prime}\Gamma^{\prime\prime-1}%
448: \delta^{\prime}\right] ,$\\
449: $\psi=\left( \Gamma^{\prime\prime}-\Gamma^{\prime\ast}\Gamma^{-1}%
450: \Gamma^{\prime}\right) ^{-1}\left[ \delta^{\prime}-\Gamma^{\prime\ast}%
451: \Gamma^{-1}\delta\right] .$%
452: \end{tabular}
453: \ \ \ \ \ \ \ \ \right. \label{soluce}%
454: \end{equation}
455: Let us denote
456: \begin{align*}
457: S_{\phi} & =\Gamma-\Gamma^{\prime}\Gamma^{\prime\prime-1}\Gamma^{\prime\ast
458: },\\
459: S_{\psi} & =\Gamma^{\prime\prime}-\Gamma^{\prime\ast}\Gamma^{-1}%
460: \Gamma^{\prime}.
461: \end{align*}
462: The reader should note two crucial facts. On the one hand $\Gamma^{-1}$ and
463: $\Gamma^{\prime\prime-1}$ are unbouded operators but closed graphs argument
464: ensure that $\Gamma^{\prime}\Gamma^{\prime\prime-1}\delta^{\prime}$ and
465: $\Gamma^{\prime\ast}\Gamma^{-1}\delta$ exist in $W$ and $L$ respectively. On
466: the other hand $\delta-\Gamma^{\prime}\Gamma^{\prime\prime-1}\delta^{\prime}$
467: (resp. $\delta^{\prime}-\Gamma^{\prime\ast}\Gamma^{-1}\delta$) belong to the
468: domain of the unbounded operator $S_{\phi}^{-1}$ (resp. $S_{\psi}^{-1}$) which
469: also ensures the finiteness of both solutions given in the display above.
470:
471: Finding approximations to the solutions of (\ref{soluce}) is known in the
472: mathematical literature as "solving a linear inverse problem". The book by
473: Tikhonov and Arsenin (1977) -as many other references therein- is devoted to
474: this theory well-known in image reconstruction. The unboundedness of $S_{\phi
475: }^{-1}$ may cause large variation of $S_{\phi}^{-1}x$ even for small
476: variations of $x$. This lack of stability turns out to damage, as well as the
477: traditional "curse of dimensionality", the rates of convergence of our estimates.
478:
479: Unfortunately we cannot simply replace "theoretical" operators and vectors by
480: their empirical estimates because $\Gamma_{n}$ and $\Gamma_{n}^{\prime\prime}$
481: are not invertible. Indeed they are finite-rank operators (for example the
482: image of $\Gamma$ is \textrm{span}$\left( X_{1},...,X_{n}\right) $) hence
483: not even injective. We are classically going to add a small perturbation to
484: regularize $\Gamma_{n}$ and $\Gamma_{n}^{\prime\prime}$ (see Tikhonov and
485: Arsenin (1977)) and another one for $S_{\phi}^{-1}$ and make them invertible.
486: At last $\Gamma^{-1}$ is approximated by $\Gamma_{n}^{\dag}=\left( \Gamma
487: _{n}+\alpha_{n}I\right) ^{-1},$ $\Gamma^{\prime\prime-1}$ by $\Gamma
488: _{n}^{\prime\prime\dag}=\left( \Gamma_{n}^{\prime\prime}+\alpha_{n}I\right)
489: ^{-1}$ and $S_{\phi}^{-1}$ by $\left( S_{n,\phi}+\beta_{n}I\right) ^{-1}$
490: where%
491: \[
492: S_{n,\phi}=\Gamma_{n}-\Gamma_{n}^{\prime}\left( \Gamma_{n}^{\prime
493: \prime\dagger}\right) \Gamma_{n}^{\prime\ast}.
494: \]
495: and $\alpha_{n}>0,$ $\beta_{n}>0.$ We also set :%
496: \begin{align}
497: S_{n,\psi} & =\Gamma_{n}^{\prime\prime}-\Gamma_{n}^{\prime\ast}\left(
498: \Gamma_{n}^{\dagger}\right) \Gamma_{n}^{\prime},\label{Snpsi}\\
499: u_{n,\phi} & =\delta_{n}-\Gamma_{n}^{\prime}\left( \Gamma_{n}^{\prime
500: \prime\dagger}\right) \delta_{n}^{\prime},\label{Tnphi}\\
501: u_{n,\psi} & =\delta_{n}^{\prime}-\Gamma_{n}^{\prime\ast}\left( \Gamma
502: _{n}^{\dagger}\right) \delta_{n}. \label{Tnpsi}%
503: \end{align}
504:
505:
506: In the sequel we will assume that both strictly positive sequences $\alpha
507: _{n}$ and $\beta_{n}$ decay to zero in order to get the asymptotic convergence
508: of the estimates.
509:
510: \begin{definition}
511: The estimate of the couple $\left( \phi,\psi\right) $ is $\left(
512: \widehat{\phi}_{n},\widehat{\psi}_{n}\right) $ based on (\ref{soluce}) and
513: defined by :%
514: \begin{equation}
515: \left\{
516: \begin{tabular}
517: [c]{l}%
518: $\widehat{\phi}_{n}=\left( S_{n,\phi}+\beta_{n}I\right) ^{-1}u_{n,\phi
519: },\smallskip$\\
520: $\widehat{\psi}_{n}=\left( S_{n,\psi}+\beta_{n}I\right) ^{-1}u_{n,\psi}.$%
521: \end{tabular}
522: \ \right. \label{estim}%
523: \end{equation}
524: The predictor is defined as%
525: \[
526: \widehat{y}_{n+1}=\left\langle \widehat{\phi}_{n},X_{n+1}\right\rangle
527: _{W}+\left\langle \widehat{\psi}_{n},X_{n+1}^{\prime}\right\rangle _{L}.
528: \]
529:
530: \end{definition}
531:
532: \section{Main results and comments}
533:
534: In Mas, Pumo (2006) the authors obtained convergence in probability for their
535: estimates in a quite different model. We are now in position to assess deeper
536: results. Mean square prediction error is indeed given an asymptotic
537: development depending on both smoothing sequences $\alpha_{n}$ and $\beta_{n}$.
538:
539: Before stating the main result of this article, we give and comment the next
540: and last assumption :%
541: \begin{equation}
542: \mathbf{A4}:\left\{
543: \begin{tabular}
544: [c]{l}%
545: $\left\Vert \Gamma^{-1/2}\phi\right\Vert _{W}<+\infty$\\
546: $\left\Vert \left( \Gamma^{\prime\prime}\right) ^{-1/2}\psi\right\Vert
547: _{L}<+\infty$%
548: \end{tabular}
549: \ \ \ \ \ \right. \label{A4}%
550: \end{equation}
551:
552:
553: For the definition of $\Gamma^{-1/2}$ and $\Gamma^{\prime\prime-1/2}$ we refer
554: to (\ref{racop}). Let us explain briefly what both conditions in (\ref{A4})
555: mean. To that aim we rewrite the first by developing $\Gamma^{-1/2}\phi$ in a
556: basis of eigenvectors of $\Gamma,$ say $u_{p}$%
557: \[
558: \Gamma^{-1/2}\phi=\sum_{p=1}^{+\infty}\dfrac{\left\langle \phi,u_{p}%
559: \right\rangle }{\sqrt{\lambda_{p}}}u_{p}%
560: \]
561: hence%
562: \[
563: \left\Vert \Gamma^{-1/2}\phi\right\Vert _{W}^{2}=\sum_{p=1}^{+\infty}%
564: \dfrac{\left\langle \phi,u_{p}\right\rangle ^{2}}{\lambda_{p}}%
565: \]
566: The first part of assumption $\mathbf{A4}$ tells us that "$\left\langle
567: \phi,u_{p}\right\rangle $ should tend to zero quickly enough with respect to
568: $\lambda_{p}$". In other words $\phi$ should belong to an ellipso\"{\i}d of
569: $W$ which may be more or less "flat" depending on the rate of decay of the
570: $\lambda_{p}$'s to zero. Assumption $\mathbf{A4}$ is in fact \textbf{a
571: regularity condition} on functions $\phi$ and $\psi$ : function $\phi$ (resp.
572: $\psi$) should be smoother than $X$ (resp. $X^{\prime}$).\bigskip
573:
574: We could try and state convergence results for $\widehat{\phi}_{n}$ and
575: $\widehat{\psi}_{n}$ separatedly but it turns out that :
576:
577: \begin{itemize}
578: \item The real statistical interest of the model relies on its predictive
579: power. The statistician is mainly interested in $\widehat{y}_{n+1},$ not in
580: $\widehat{\phi}_{n}$ and $\widehat{\psi}_{n}$ in a first attempt. The issue of
581: goodness of fit tests (involving $\phi$ and $\psi$ alone) is beyond the scope
582: of this article.
583:
584: \item Considering the mean square norm of $\left\langle \widehat{\phi}%
585: _{n},X_{n+1}\right\rangle _{W}$ (instead of $\widehat{\phi}_{n}$ or even of
586: $\left\langle \widehat{\phi}_{n},x\right\rangle _{W}$ for a nonrandom $x$) has
587: a smoothing effect on our estimates and partially counterbalance the side
588: effects of the underlying inverse problem as will be seen within the proofs
589: (especially along Lemma \ref{placebo}).
590: \end{itemize}
591:
592: Turning to $\widehat{y}_{n+1},$ the next question is : what should we compare
593: $\widehat{y}_{n+1}$ with~? The right answer is not $y_{n+1}.$ Obviously we
594: could, but it is also plain that, due to the random $\varepsilon_{n+1}$ the
595: best possible prediction for $y_{n+1}$ knowing $X_{n+1}$ (or even the "past"
596: i.e. $X_{1},...,X_{n}$) is the conditional expectation :%
597: \[
598: y_{n+1}^{\ast}=\mathbb{E}\left( y_{n+1}|X_{1},...,X_{n+1}\right)
599: =\left\langle \phi,X_{n+1}\right\rangle _{W}+\left\langle \psi,X_{n+1}%
600: ^{\prime}\right\rangle .
601: \]
602: We are now ready to state the main theoretical result of this article.
603:
604: \begin{theorem}
605: \label{conv}When assumptions $\mathbf{A1-A4}$ hold the following expansion is
606: valid for the prediction mean square error :%
607: \[
608: \mathbb{E}\left( \widehat{y}_{n+1}-y_{n+1}^{\ast}\right) ^{2}=O\left(
609: \frac{\beta^{2}}{\alpha^{2}}\right) +O\left( \frac{1}{\alpha^{2}\beta^{2}%
610: n}\right)
611: \]
612:
613: \end{theorem}
614:
615: \begin{remark}
616: Replacing $y_{n+1}^{\ast}$ with $y_{n+1}$ is still possible. We may easily
617: prove that~:%
618: \[
619: \mathbb{E}\left( \widehat{y}_{n+1}-y_{n+1}\right) ^{2}=\mathbb{E}\left(
620: \widehat{y}_{n+1}-y_{n+1}^{\ast}\right) ^{2}+\sigma_{\varepsilon}^{2}.
621: \]
622:
623: \end{remark}
624:
625: \begin{corollary}
626: From Theorem \ref{conv} above an optimal choice for $\beta$ is $\beta^{\ast
627: }\asymp n^{-1/4},$ then the convergence rate is :%
628: \[
629: \mathbb{E}\left( \widehat{y}_{i}-y_{i}^{\ast}\right) ^{2}=O\left( \frac
630: {1}{\alpha^{2}n^{1/2}}\right)
631: \]
632: and may be quite close from $1/n^{1/2}$.
633: \end{corollary}
634:
635: The proof of the Corollary will be omitted. Studying the optimality of this
636: rate of convergence over the classes of functions defined by $\mathbf{A4}$ is
637: beyond the scope of this article but could deserve more attention.
638:
639: \begin{remark}
640: Originally the linear model (\ref{modele}) is subject to serious
641: multicolinearity troubles since $X_{n}^{\prime}=DX_{n}.$ Even if the curve
642: $X_{n}^{\prime}$ usually looks quite different from $X_{n},$ there is a total
643: stochastic dependence between them. The method used in this article to tackle
644: this problem (as well as the intrinsic "inverse problem" aspects related to
645: the inversion of the covariance operators $\Gamma$ and $\Gamma^{\prime\prime}%
646: $) is new up to the authors' knowledge. As it can be seen through above at
647: display (\ref{estim}) or in the proofs below, it relies on a double
648: penalization technique first by the index $\alpha_{n}$ then by $\beta_{n}$
649: linking both indexes in order to suppress the bias terms asymptotically .
650: \end{remark}
651:
652: \section{An application to spectrometric data}
653:
654: In this section we will present an application of the Functional Linear
655: Regression with Derivatives (FLRD) introduced in this paper to a spectroscopic
656: calibration problem. Quantitative NIR (near-infrared) spectroscopy is used to
657: analyze food and agricultural materials. The NIR spectrum of a sample is a
658: continuous curve giving the absorption, that is $\log_{10}1/R$ where $R$ is
659: the reflection of the sample, against wavelength measured in nanometers (nm).
660:
661: In the \texttt{cookie} example considered here the aim is to predict the
662: percentage of each ingredient $y$ given the NIR spectrum $x$ of the sample
663: (see Osborne et al. (1984) for a full description of the experiment). The
664: constituents under investigation are: fat, sucrose, dry flour, and water.
665: There were 39 samples in the calibration set, sample number 23 having been
666: excluded from the original 40 as an outlier, and a further validation set with
667: 31 samples, again after the exclusion of one outlier.
668:
669: An NIR reflectance spectrum is available for each dough. The original spectral
670: data consists of 700 points measured from 1100 to 1498 nm in steps of 2 nm.
671: Following Brown et al. (2001) we reduced the number of spectral points to 256
672: by considering only the spectral range 1380-2400 nm in step of 4 nm. Samples
673: of centered spectra are plotted in Figure \ref{Cookie_spectres}.\bigskip\
674: %TCIMACRO{\FRAME{ftbpFU}{4.9035in}{3.1099in}{0pt}{\Qcb{Centered original
675: %spectra of four samples (256 measures)}}{\Qlb{Cookie_spectres}}%
676: %{cookie_spectres.eps}{\special{ language "Scientific Word"; type "GRAPHIC";
677: %maintain-aspect-ratio TRUE; display "USEDEF"; valid_file "F";
678: %width 4.9035in; height 3.1099in; depth 0pt; original-width 7.6268in;
679: %original-height 4.8274in; cropleft "0"; croptop "1"; cropright "1";
680: %cropbottom "0"; filename 'cookie_spectres.eps';file-properties "XNPEU";}}}%
681: %BeginExpansion
682: \begin{figure}
683: [ptb]
684: \begin{center}
685: \includegraphics[
686: height=3.1099in,
687: width=4.9035in
688: ]%
689: {cookie_spectres.eps}%
690: \caption{Centered original spectra of four samples (256 measures)}%
691: \label{Cookie_spectres}%
692: \end{center}
693: \end{figure}
694: %EndExpansion
695:
696:
697: %\begin{figure}[h]
698: %\begin{center}
699: %\includegraphics*[scale=0.5 ]{Figures/Cookie-spectres.eps}
700: %\end{center}
701: %\caption{The spectrometric curves.}
702: %\label{Cookie_spectres}
703: %\end{figure}
704:
705:
706: A classical tool employed in the chemiometric literature for the prediction of
707: $y$ knowing the associated NIR spectra $(x_{j},j=1,\ldots,256)$ is the linear
708: model:%
709: \begin{equation}
710: y=\sum_{j=1,256}\theta_{j}x_{j}+\epsilon\label{Mult.Lin.Mod}%
711: \end{equation}
712: The problem then is to use the calibration data to estimate the unknown
713: parameters $\theta_{j}$. Clearly in this application since $39\ll256$ the
714: ordinary least squares fails and many authors proposed to use alternative
715: methods to tackle the problem: principal component regression (PCR) or partial
716: least squares regression (PLS). We invite the reader to look at the paper of
717: Frank and Friedman (1993) for a statistical view of some chemiometrics
718: regression tools.
719:
720: Following an idea of Hastie and Mallows, in their discussion of Frank and
721: Friedman's paper, we consider a spectrum as a functional observation. The
722: functional Linear Regression (FLR) corresponding to the model
723: \ref{Mult.Lin.Mod} defined above is:%
724: \[
725: y=\int_{\delta}x(t)\theta(t)dt+\varepsilon
726: \]
727: where $y$ is a scalar random variable, $x$ a real function defined on
728: $\delta=[1100,2400]$\ and $\theta(t)$ the unknown parameter function. Brown et
729: al. (2001), Ferraty and Vieu (2003), Marx and Eilers (2002) or Amato et al.
730: (2006) used such a model for a prediction problem with spectrometric data.
731:
732: The model FLRD introduced in this paper can be written as:%
733: \[
734: y=\int_{\delta}x(t)\phi(t)dt+\int_{\delta}x^{\prime}(t)\psi(t)dt+\varepsilon
735: \]
736: where $\phi(t)$ and $\psi(t)$ are unknown functions (see display
737: (\ref{modele}) for an equivalent definition). In this paragraph we compare the
738: performance of PCR, PLS, FLR, FLRD, Spline Smoothing model proposed by Cardot,
739: Ferraty and Sarda (2006) and Bayes wavelet predictions proposed by Brown et
740: al. (2001).
741:
742: We used the calibration data set for the estimation of parameter functions
743: $\phi(t)$ and $\psi(t)$ and validation data for calculation of the MSEP (Mean
744: Squared Error of Predictions):%
745: \[
746: MSEP=\frac{1}{31}\sum_{j=1}^{31}(y_{j}-\hat{y}_{j})^{2}%
747: \]
748: where $\hat{y}_{j}$\ is the prediction of $y_{j}$ obtained by the model with
749: estimated parameters. The choice of the parameters $\alpha$ and $\beta$ is
750: crucial for the prediction model. We used a cross-validation approach based on
751: the evaluation of the standard error of prediction $CVMSEP$:
752: \[
753: CVMSEP(\alpha,\beta)=\frac{1}{39}\sum_{i=1}^{39}[\frac{1}{38}\sum
754: \nolimits_{j=1}^{38}(y_{i}^{c}-\hat{y}_{j}^{c}(i;\alpha,\beta))^{2}],
755: \]
756: where $\hat{y}_{j}^{c}(i;\alpha,\beta)$ denotes the prediction of $y_{j}^{c}%
757: $\ in the calibration set without sample $i$. Results for different methods of
758: prediction of four ingredients are displayed in Table \ref{Table.MSEP}. We
759: used B-spline basis ($k=100$) for obtaining predictions with Spline Smoothing,
760: Spline Ridge RLF and Spline RLFD methods. For each of those methods we give
761: the values of the smoothing or penalty parameters based on an analogous
762: cross-validation approach.
763:
764: \begin{table}[h]
765: \begin{center}%
766: \begin{tabular}
767: [c]{|l|cccc|}\hline
768: & \multicolumn{4}{|c|}{MSE Validation}\\
769: Method and parameters & Fat & Sugar & Flour & Water\\\hline
770: PLS & 0.151 & 0.583 & 0.375 & 0.105\\
771: PCR & 0.160 & 0.614 & 0.388 & 0.106\\
772: Spline Smoothing ($k_{n}=8$) & 0.546 & 0.471 & 2.226 & 0.183\\
773: Spline Ridge FLR ($\beta=0.00002$) & 0.044 & 0.494 & 0.318 & 0.087\\
774: Spline FLRD ($\alpha=0.07,\beta=0.15$) & 0.092 & 0.450 & 0.332 & 0.069\\
775: Bayes Wavelet & 0.063 & 0.449 & 0.348 & 0.050\\\hline
776: \end{tabular}
777: \end{center}
778: \caption{$MSEP$ criterion for all models (see Brown et al. for results of PLS,
779: PCR and Bayes wavelet methods).}%
780: \label{Table.MSEP}%
781: \end{table}
782:
783: We note that functional approaches work better then PLS or PCR methods for the
784: four predicted variables with respect to $MSEP$ criterion. Our simulation, as
785: noted also by Marx and Eilers (2002), show that functional methods lead to
786: more stable prediction. The Spline FLRD method produces in general equivalent
787: results in terms of predictions with the best methods presented in table
788: \ref{Table.MSEP}.
789:
790: \bigskip
791:
792: \section{Proofs}
793:
794: In the sequel $M$ and $M^{\prime}$ will stand for constants.
795:
796: Let $S$ and $T$ be two selfadjoint linear operators on a Hilbert space $H,$ we
797: denote $T\ll S$ whenever for all $x$ in $H,$ $\left\langle Tx,x\right\rangle
798: \leq\left\langle Sx,x\right\rangle $ then $\left\Vert T\right\Vert _{\infty
799: }\leq\left\Vert S\right\Vert _{\infty}$.
800:
801: The norm in the space $L^{2}\left( B\right) $ where $\left( B,\left\Vert
802: \cdot\right\Vert _{B}\right) $ is a Banach space is defined the following way
803: : let $X$ be a random element in the Banach space $B,$ then
804: \[
805: \left\Vert X\right\Vert _{L^{2}\left( B\right) }=\left( E\left\Vert
806: X\right\Vert _{B}^{2}\right) ^{1/2}%
807: \]
808: When the notation is not ambiguous we systematically drop the index $B$ i.e :
809: $\left\Vert X\right\Vert _{L^{2}}=\left( E\left\Vert X\right\Vert _{B}%
810: ^{2}\right) ^{1/2}$.
811:
812: \subsection{Preliminary facts :}
813:
814: In order to gain some clarity in the proofs and to alleviate them we first
815: list a few results stemming from operator or probabillity theory.
816:
817: \textbf{Fact 1}: If $T$ is a positive operator (either random or not),
818: $T+\gamma I$ is invertible for all $\gamma>0$ with bounded inverse and
819: $\left\Vert \left( T+\gamma I\right) ^{-1}\right\Vert _{\infty}\leq
820: \gamma^{-1}$. Hence%
821: \begin{equation}
822: \left\Vert \Gamma_{n}^{\dag}\right\Vert _{\infty}=\left\Vert \Gamma^{\dag
823: }\right\Vert _{\infty}=\left\Vert \Gamma_{n}^{\prime\prime\dag}\right\Vert
824: _{\infty}=\left\Vert \Gamma^{\prime\prime\dag}\right\Vert _{\infty}%
825: =\alpha^{-1} \label{norm.res}%
826: \end{equation}
827:
828:
829: \textbf{Fact 2}: As a consquence of assumption $\mathbf{A1}$ and of the strong
830: law of large numbers for Hilbert valued random elements (see Ledoux, Talagrand
831: (1991) Chapter 7),%
832: \[
833: T_{n}\underset{n\rightarrow+\infty}{\rightarrow}T\quad a.s.
834: \]
835: whenever $T_{n}=\Gamma_{n},\Gamma_{n}^{\prime},\Gamma_{n}^{\prime\ast}%
836: ,\Gamma_{n}^{\prime\prime}$ (resp. $T=\Gamma,\Gamma^{\prime},\Gamma
837: ^{\prime\ast},\Gamma^{\prime\prime}$) since all theses random operators may be
838: rewritten as sums of i.i.d. random variables. These sequences of random
839: operators are almost surely bounded%
840: \begin{equation}
841: \sup_{n}\left\Vert T_{n}\right\Vert _{\infty}\leq M\quad a.s.
842: \label{norm.delta}%
843: \end{equation}
844: which also means that
845: \begin{equation}
846: \max\left( \sup_{n}\left\Vert \delta_{n}\right\Vert _{W},\sup_{n}\left\Vert
847: \delta_{n}^{\prime}\right\Vert _{L}\right) \leq M^{\prime} \label{asd}%
848: \end{equation}
849: since (for instance) $\delta_{n}=\Gamma_{n}\phi+\Gamma_{n}^{\prime}\psi+e_{n}$
850: where $e_{n}$ is again a sum of i.i.d random elements :%
851: \[
852: e_{n}=\frac{1}{n}\sum_{k=1}^{n}X_{k}\varepsilon_{k}%
853: \]
854: We also set%
855: \[
856: e_{n}^{\prime}=\frac{1}{n}\sum_{k=1}^{n}X_{k}^{\prime}\varepsilon_{k}%
857: \]
858: (see below for details).
859:
860: \textbf{Fact 3}: The Central Limit Thorem in Hilbert spaces (or standards
861: results on rates of convergence for Hilbert valued random elements in square
862: norm) provide a rate in the $L^{2}$ convergence of several random variables of
863: interest in the proofs. See for instance Ledoux, Talagrand (1991) or Bosq
864: (2000) . Whenever $T_{n}=\Gamma_{n},\Gamma_{n}^{\prime},\Gamma_{n}^{\prime
865: \ast},\Gamma_{n}^{\prime\prime}$ (resp. $T=\Gamma,\Gamma^{\prime}%
866: ,\Gamma^{\prime\ast},\Gamma^{\prime\prime}$) we have $\mathbb{E}\left\Vert
867: T_{n}-T\right\Vert _{\mathcal{HS}}^{2}=O\left( \frac{1}{n}\right) $ hence
868: \begin{equation}
869: \left\Vert T_{n}-T\right\Vert _{L^{2}\left( \mathcal{HS}\right) }=O\left(
870: \dfrac{1}{\sqrt{n}}\right) \label{L2}%
871: \end{equation}
872: since all theses random operators may be rewritten as sums of i.i.d. random variables.
873:
874: We begin with proving Proposition \ref{Ident}.
875:
876: \textbf{Proof of Proposition \ref{Ident} :}
877:
878: The method of the proof may be adapted from the model studied in Mas, Pumo
879: (2006). The couple $\left( \phi,\psi\right) $ will be identified whenever,
880: for any other couple $\left( \phi_{a},\psi_{a}\right) $, if
881: \[
882: \left\{
883: \begin{tabular}
884: [c]{l}%
885: $\delta=\Gamma\phi+\Gamma^{\prime}\psi=\Gamma\phi_{a}+\Gamma^{\prime}\psi
886: _{a},$\\
887: $\delta^{\prime}=\Gamma^{\prime\ast}\phi+\Gamma^{\prime\prime}\psi
888: =\Gamma^{\prime\ast}\phi_{a}+\Gamma^{\prime\prime}\psi_{a}.$%
889: \end{tabular}
890: \right.
891: \]
892: $\left( \phi_{a},\psi_{a}\right) =\left( \phi,\psi\right) $. This will be
893: true if%
894: \[
895: \left\{
896: \begin{tabular}
897: [c]{l}%
898: $\Gamma\left( \phi-\phi_{a}\right) +\Gamma^{\prime}\left( \psi-\psi
899: _{a}\right) =0,$\\
900: $\Gamma^{\prime\ast}\left( \phi-\phi_{a}\right) +\Gamma^{\prime\prime
901: }\left( \psi-\psi_{a}\right) =0.$%
902: \end{tabular}
903: \right.
904: \]
905:
906:
907: This means that the couple $\left( \phi-\phi_{a},\psi-\psi_{a}\right) $
908: belongs to the kernel of the linear operator defined blockwise on $W\times L$
909: by :%
910: \[
911: \left(
912: \begin{array}
913: [c]{cc}%
914: \Gamma & \Gamma^{\prime}\\
915: \Gamma^{\prime\ast} & \Gamma^{\prime\prime}%
916: \end{array}
917: \right) .
918: \]
919: As $\Gamma^{\prime\ast}=D\Gamma$ and $\Gamma^{\prime\prime}=D\Gamma^{\prime}$,
920: the Proposition will be proved if the blockwise operator defined on $W\times
921: L$ and with values in $W$ :%
922: \[
923: \left(
924: \begin{array}
925: [c]{cc}%
926: \Gamma & \Gamma^{\prime}%
927: \end{array}
928: \right) =\left(
929: \begin{array}
930: [c]{cc}%
931: \Gamma & \Gamma D^{\ast}%
932: \end{array}
933: \right)
934: \]
935: is one to one. It is plain that the kernel of this operator is precisely the
936: space $\mathcal{N}$ that appears at display (\ref{fluke}).
937:
938: This finishes the proof of the Proposition.
939:
940: The next two general Propositions are proved for further purpose.
941:
942: \begin{proposition}
943: \label{lodge}%
944: \begin{align*}
945: \sup_{n}\left\Vert \left( \Gamma_{n}^{\prime\prime\dag}\right) ^{1/2}%
946: \Gamma_{n}^{\prime\ast}\right\Vert _{\infty} & <M\quad a.s.,\\
947: \sup_{n}\left\Vert \Gamma_{n}^{\prime}\left( \Gamma_{n}^{\prime\prime\dag
948: }\right) ^{1/2}\right\Vert _{\infty} & <M\quad a.s.,\\
949: \sup_{n}\left\Vert \left( \Gamma^{\prime\prime\dag}\right) ^{1/2}%
950: \Gamma^{\prime\ast}\right\Vert _{\infty} & <M,\\
951: \sup_{n}\left\Vert \Gamma^{\prime}\left( \Gamma^{\prime\prime\dag}\right)
952: ^{1/2}\right\Vert _{\infty} & <M.
953: \end{align*}
954:
955: \end{proposition}
956:
957: \begin{proof}
958: We prove only the first bound since the method may be copied for the other
959: ones. Set $R_{n}=D\Gamma_{n}^{1/2}$ then :%
960: \begin{align*}
961: \Gamma_{n}^{\prime\prime} & =R_{n}R_{n}^{\ast},\\
962: \left( \Gamma_{n}^{\prime\prime\dag}\right) ^{1/2}\Gamma_{n}^{\prime\ast}
963: & =\left( R_{n}R_{n}^{\ast}+\alpha I\right) ^{-1/2}R_{n}\Gamma_{n}^{1/2}.
964: \end{align*}
965: At last,%
966: \[
967: \left\Vert \left( \Gamma_{n}^{\prime\prime\dag}\right) ^{1/2}\Gamma
968: _{n}^{\prime\ast}\right\Vert _{\infty}\leq\left\Vert \left( R_{n}R_{n}^{\ast
969: }+\alpha I\right) ^{-1/2}R_{n}\right\Vert _{\infty}\left\Vert \Gamma
970: _{n}^{1/2}\right\Vert _{\infty}.
971: \]
972: It is plain that%
973: \[
974: \sup_{n}\left\Vert \Gamma_{n}^{1/2}\right\Vert _{\infty}\leq M\quad a.s.
975: \]
976: If the Schmidt decomposition of $R_{n}$ is :%
977: \[
978: R_{n}=\sum_{k\in\mathbb{N}}\mu_{k,n}\left( u_{k,n}\otimes v_{k,n}\right) ,
979: \]
980: $(u_{k,n}\in W,\ v_{k,n}\in L)$ it is simple algebra to get :%
981: \begin{equation}
982: \left( R_{n}R_{n}^{\ast}+\alpha I\right) ^{-1/2}R_{n}=\sum_{k\in\mathbb{N}%
983: }\frac{\mu_{k,n}}{\sqrt{\mu_{k,n}^{2}+\alpha}}\left( u_{k,n}\otimes
984: v_{k,n}\right) \label{mano}%
985: \end{equation}
986: which yields $\left\Vert \left( R_{n}R_{n}^{\ast}+\alpha I\right)
987: ^{-1/2}R_{n}\right\Vert _{\infty}=\sup_{k}\left\{ \frac{\mu_{k,n}}{\sqrt
988: {\mu_{k,n}^{2}+\alpha}}\right\} \leq1.$
989: \end{proof}
990:
991: \begin{proposition}
992: \label{KT}%
993: \[
994: \left\Vert \left( S_{n}+\beta I\right) ^{-1}\right\Vert _{\infty}\leq
995: \dfrac{1}{\beta}.
996: \]
997:
998: \end{proposition}
999:
1000: \begin{proof}
1001: The proof of this Lemma is similar to Lemma 7.4 in Mas, Pumo (2006). It was
1002: then proved for $S$ instead of $S_{n}$ and all operators should be changed to
1003: their empirical counterparts (e.g : $\Gamma_{n}$ insted of $\Gamma$). We give
1004: a sketch of it. The proof relies on the Schmidt decomposition of $S_{n}.$ One
1005: would get%
1006: \[
1007: S_{n}=\Gamma_{n}^{1/2}\Lambda_{n}\left( \alpha\right) \Gamma_{n}^{1/2}%
1008: \]
1009: where $\Lambda_{n}\left( \alpha\right) $ and $\Gamma_{n}^{1/2}$ are
1010: symmetric positive operators, which implies that $S_{n}$ itself is positive.
1011: It suffices then to apply \textbf{Fact 2 }(see the "Preliminary facts"
1012: subsection) to get the desired result.
1013: \end{proof}
1014:
1015: \subsection{Outline of the proof of Theorem \ref{conv} :}
1016:
1017: The following bound is valid :%
1018: \begin{align*}
1019: & \left[ \widehat{y}_{n+1}-\left( \left\langle \phi,X_{n+1}\right\rangle
1020: _{W}+\left\langle \psi,X_{n+1}^{\prime}\right\rangle _{L}\right) \right]
1021: ^{2}\\
1022: & =\left( \left\langle \phi-\widehat{\phi},X_{n+1}\right\rangle
1023: _{W}+\left\langle \psi-\widehat{\psi},X_{n+1}^{\prime}\right\rangle
1024: _{L}\right) ^{2}\\
1025: & \leq2\left[ \left\langle \phi-\widehat{\phi},X_{n+1}\right\rangle _{W}%
1026: ^{2}+\left\langle \psi-\widehat{\psi},X_{n+1}^{\prime}\right\rangle _{L}%
1027: ^{2}\right] .
1028: \end{align*}
1029: Then
1030: \begin{align*}
1031: \mathbb{E}\left\langle \phi-\widehat{\phi},X_{n+1}\right\rangle _{W}^{2} &
1032: =\mathbb{E}\left[ \mathbb{E}\left\langle \phi-\widehat{\phi},X_{n+1}%
1033: \right\rangle _{W}^{2}|X_{1},...,X_{n}\right] \\
1034: & =\mathbb{E}\left[ \mathbb{E}\left\langle \phi-\widehat{\phi}%
1035: ,X_{n+1}\right\rangle _{W}^{2}|\widehat{\phi}\right] \\
1036: & =\mathbb{E}\left[ \left\Vert \Gamma^{1/2}\left( \phi-\widehat{\phi
1037: }\right) \right\Vert _{W}^{2}\right]
1038: \end{align*}
1039: Similarly,%
1040: \[
1041: \mathbb{E}\left\langle \psi-\widehat{\psi},X_{n+1}^{\prime}\right\rangle
1042: _{L}^{2}=\mathbb{E}\left[ \left\Vert \Gamma^{\prime\prime1/2}\left(
1043: \psi-\widehat{\psi}\right) \right\Vert _{L}^{2}\right]
1044: \]
1045: Both preceding equations feature similar expressions. We focus on the term
1046: involving $\phi$ ; we will prove that :%
1047: \[
1048: \mathbb{E}\left[ \left\Vert \Gamma^{1/2}\left( \phi-\widehat{\phi}\right)
1049: \right\Vert _{W}^{2}\right] =O\left( \frac{\beta^{2}}{\alpha^{2}}\right)
1050: +O\left( \frac{1}{\alpha^{2}\beta^{2}n}\right) .
1051: \]
1052: Within the proof the reader will easily be convinced that the method would
1053: lead to an analogous result for the term with $\psi.$ From now in order to
1054: alleviate notations we drop the index $\phi$ in $S_{n,\phi}$ and $u_{n,\phi}$.
1055: The sequences $\left( \alpha_{n}\right) _{n\in\mathbb{N}}$ and $\left(
1056: \beta_{n}\right) _{n\in\mathbb{N}}$ will be denoted $\alpha$ and $\beta$
1057: respectively and for short.
1058:
1059: We start from%
1060: \begin{align*}
1061: \widehat{\phi}_{n} & =\left( \Gamma_{n}-\Gamma_{n}^{\prime}\Gamma
1062: _{n}^{\prime\prime\dag}\Gamma_{n}^{\prime\ast}+\beta I\right) ^{-1}\left(
1063: \delta_{n}-\Gamma_{n}^{\prime}\Gamma_{n}^{\prime\prime\dag}\delta_{n}^{\prime
1064: }\right) \\
1065: & =\left( S_{n}+\beta I\right) ^{-1}u_{n}\\
1066: \phi & =\left( \Gamma-\Gamma^{\prime}\Gamma^{\prime\prime\dag}\Gamma
1067: ^{\prime\ast}\right) ^{-1}\left( \delta-\Gamma^{\prime}\Gamma^{\prime
1068: \prime\dag}\delta^{\prime}\right) \\
1069: & =S^{-1}u
1070: \end{align*}
1071:
1072:
1073: where we recall that :%
1074: \begin{align*}
1075: u & =\delta-\Gamma^{\prime}\Gamma^{\prime\prime\dag}\delta^{\prime},\\
1076: u_{n} & =\delta_{n}-\Gamma_{n}^{\prime}\Gamma_{n}^{\prime\prime\dag}%
1077: \delta_{n}^{\prime},\\
1078: S & =\Gamma-\Gamma^{\prime}\Gamma^{\prime\prime\dag}\Gamma^{\prime\ast},\\
1079: S_{n} & =\Gamma_{n}-\Gamma_{n}^{\prime}\Gamma_{n}^{\prime\prime\dag}%
1080: \Gamma_{n}^{\prime\ast}.
1081: \end{align*}
1082:
1083:
1084: The proof relies on the following decomposition :%
1085: \begin{align}
1086: \widehat{\phi}_{n}-\phi & =\left( S_{n}+\beta I\right) ^{-1}\left(
1087: u_{n}-u\right) +\left( \left( S_{n}+\beta I\right) ^{-1}-S^{-1}\right)
1088: u\nonumber\\
1089: & =\left( S_{n}+\beta I\right) ^{-1}\left( u_{n}-u\right) +\left(
1090: S_{n}+\beta I\right) ^{-1}\left( S-S_{n}-\beta I\right) S^{-1}u\nonumber\\
1091: & =A_{n}+B_{n}+C_{n} \label{decomp}%
1092: \end{align}
1093: where%
1094: \begin{align}
1095: A_{n} & =\left( S_{n}+\beta I\right) ^{-1}\left( u_{n}-u\right)
1096: \label{T1}\\
1097: B_{n} & =\left( S_{n}+\beta I\right) ^{-1}\left( S-S_{n}\right)
1098: \phi\label{T2}\\
1099: C_{n} & =\beta\left( S_{n}+\beta I\right) ^{-1}\phi\label{T3}%
1100: \end{align}
1101: Along the forthcoming Lemmas we determine rates of convergence for these three
1102: terms. We will prove that the rate of decrease to zero in $L^{2}$ norm is
1103: $\left( \alpha\beta\sqrt{n}\right) ^{-1}$ for $A_{n}$ and $B_{n}.$ The rest
1104: of the proof of the main Theorem is postponed to the end of the next and last subsection.
1105:
1106: \subsection{Proof of the main Theorem}
1107:
1108: The first Lemma gives a reta of convergence for $S_{n}-S$.
1109:
1110: \begin{lemma}
1111: \label{cran}The following holds :%
1112: \[
1113: S_{n}-S=\Gamma_{n}-\Gamma_{n}^{\prime}\Gamma_{n}^{\prime\prime\dag}\Gamma
1114: _{n}^{\prime\ast}-\Gamma+\Gamma^{\prime}\Gamma^{\prime\prime\dag}%
1115: \Gamma^{\prime\ast}=O_{L^{2}}\left( \dfrac{1}{\alpha\sqrt{n}}\right)
1116: \]
1117:
1118: \end{lemma}
1119:
1120: \begin{proof}
1121: First of all by (\ref{L2}) :%
1122: \[
1123: \left\Vert \Gamma_{n}-\Gamma\right\Vert _{L^{2}\left( \mathcal{HS}\right)
1124: }=O\left( \dfrac{1}{\sqrt{n}}\right)
1125: \]
1126: We focus on%
1127: \begin{align*}
1128: & \Gamma_{n}^{\prime}\Gamma_{n}^{\prime\prime\dag}\Gamma_{n}^{\prime\ast
1129: }-\Gamma^{\prime}\Gamma^{\prime\prime\dag}\Gamma^{\prime\ast}\\
1130: & =\Gamma_{n}^{\prime}\Gamma_{n}^{\prime\prime\dag}\Gamma_{n}^{\prime\ast
1131: }-\Gamma^{\prime}\Gamma_{n}^{\prime\prime\dag}\Gamma_{n}^{\prime\ast}%
1132: +\Gamma^{\prime}\Gamma_{n}^{\prime\prime\dag}\Gamma_{n}^{\prime\ast}%
1133: -\Gamma^{\prime}\Gamma_{n}^{\prime\prime\dag}\Gamma^{\prime\ast}\\
1134: & +\Gamma^{\prime}\Gamma_{n}^{\prime\prime\dag}\Gamma^{\prime\ast}%
1135: -\Gamma^{\prime}\Gamma^{\prime\prime\dag}\Gamma^{\prime\ast}.
1136: \end{align*}
1137: Then dealing with each of these three terms separatedly we get%
1138: \begin{align*}
1139: \left\Vert \Gamma_{n}^{\prime}\Gamma_{n}^{\prime\prime\dag}\Gamma_{n}%
1140: ^{\prime\ast}-\Gamma^{\prime}\Gamma_{n}^{\prime\prime\dag}\Gamma_{n}%
1141: ^{\prime\ast}\right\Vert _{\infty} & \leq\left\Vert \Gamma_{n}^{\prime
1142: }-\Gamma^{\prime}\right\Vert _{\infty}\left\Vert \Gamma_{n}^{\prime\prime\dag
1143: }\Gamma_{n}^{\prime\ast}\right\Vert _{\infty}\\
1144: & \leq\left\Vert \Gamma_{n}^{\prime}-\Gamma^{\prime}\right\Vert _{\infty
1145: }\left\Vert \left( \Gamma_{n}^{\prime\prime\dag}\right) \right\Vert
1146: _{\infty}\left\Vert \Gamma_{n}^{\prime\ast}\right\Vert _{\infty}\\
1147: & \leq C\dfrac{\left\Vert \Gamma_{n}^{\prime}-\Gamma^{\prime}\right\Vert
1148: _{\infty}}{\alpha}\quad a.s.
1149: \end{align*}
1150: The last bound was derived from (\ref{norm.delta}) and (\ref{norm.res}).%
1151: \begin{align*}
1152: & \left\Vert \Gamma^{\prime}\Gamma_{n}^{\prime\prime\dag}\left( \Gamma
1153: _{n}^{\prime\ast}-\Gamma^{\prime\ast}\right) \right\Vert _{\infty}\\
1154: & \leq C\dfrac{\left\Vert \Gamma_{n}^{\prime\ast}-\Gamma^{\prime\ast
1155: }\right\Vert _{\infty}}{\alpha}\quad a.s.
1156: \end{align*}
1157: At last,%
1158: \begin{align*}
1159: \Gamma^{\prime}\left( \Gamma_{n}^{\prime\prime\dag}-\Gamma^{\prime\prime\dag
1160: }\right) \Gamma^{\prime\ast} & =\Gamma^{\prime}\Gamma_{n}^{\prime\prime
1161: \dag}\left( \Gamma^{\prime\prime}-\Gamma_{n}^{\prime\prime}\right)
1162: \Gamma^{\prime\prime\dag}\Gamma^{\prime\ast}\\
1163: & =\Gamma^{\prime}\Gamma_{n}^{\prime\prime\dag}\left( \Gamma^{\prime\prime
1164: }-\Gamma_{n}^{\prime\prime}\right) \Gamma^{\prime\prime\dag}\Gamma
1165: ^{\prime\ast}\\
1166: & =\left( \Gamma^{\prime}-\Gamma_{n}^{\prime}\right) \Gamma_{n}%
1167: ^{\prime\prime\dag}\left( \Gamma^{\prime\prime}-\Gamma_{n}^{\prime\prime
1168: }\right) \Gamma^{\prime\prime\dag}\Gamma^{\prime\ast}\\
1169: & +\Gamma_{n}^{\prime}\Gamma_{n}^{\prime\prime\dag}\left( \Gamma
1170: ^{\prime\prime}-\Gamma_{n}^{\prime\prime}\right) \Gamma^{\prime\prime\dag
1171: }\Gamma^{\prime\ast}%
1172: \end{align*}
1173: Then,%
1174: \begin{align*}
1175: & \left\Vert \Gamma^{\prime}\left( \Gamma_{n}^{\prime\prime\dag}%
1176: -\Gamma^{\prime\prime\dag}\right) \Gamma^{\prime\ast}\right\Vert _{\infty}\\
1177: & \leq\left\Vert \left( \Gamma^{\prime}-\Gamma_{n}^{\prime}\right)
1178: \Gamma_{n}^{\prime\prime\dag}\left( \Gamma^{\prime\prime}-\Gamma_{n}%
1179: ^{\prime\prime}\right) \Gamma^{\prime\prime\dag}\Gamma^{\prime\ast
1180: }\right\Vert _{\infty}\\
1181: & +\left\Vert \Gamma_{n}^{\prime}\left( \Gamma_{n}^{\prime\prime\dag
1182: }\right) ^{1/2}\right\Vert _{\infty}\left\Vert \left( \Gamma_{n}%
1183: ^{\prime\prime\dag}\right) ^{1/2}\left( \Gamma^{\prime\prime}-\Gamma
1184: _{n}^{\prime\prime}\right) \left( \Gamma^{\prime\prime\dag}\right)
1185: ^{1/2}\right\Vert _{\infty}\left\Vert \left( \Gamma^{\prime\prime\dag
1186: }\right) ^{1/2}\Gamma^{\prime\ast}\right\Vert _{\infty}%
1187: \end{align*}
1188: By Proposition \ref{lodge} the second term may be bounded by%
1189: \[
1190: C\left\Vert \left( \Gamma_{n}^{\prime\prime\dag}\right) ^{1/2}\left(
1191: \Gamma^{\prime\prime}-\Gamma_{n}^{\prime\prime}\right) \left( \Gamma
1192: ^{\prime\prime\dag}\right) ^{1/2}\right\Vert _{\infty}=O_{L^{2}}\left(
1193: \dfrac{1}{\alpha\sqrt{n}}\right)
1194: \]
1195: since
1196: \[
1197: \left\Vert \Gamma_{n}^{\prime\prime\dag1/2}\right\Vert _{\infty}=\left\Vert
1198: \Gamma^{\prime\prime\dag1/2}\right\Vert _{\infty}=\alpha^{-1/2}.
1199: \]
1200: Cauchy-Schwartz inequality yields for the first :%
1201: \begin{align*}
1202: & \mathbb{E}\left\Vert \left( \Gamma^{\prime}-\Gamma_{n}^{\prime}\right)
1203: \Gamma_{n}^{\prime\prime\dag}\left( \Gamma^{\prime\prime}-\Gamma_{n}%
1204: ^{\prime\prime}\right) \Gamma^{\prime\prime\dag}\Gamma^{\prime\ast
1205: }\right\Vert _{\infty}^{2}\\
1206: & \leq M\left( \mathbb{E}\left\Vert \left( \Gamma^{\prime}-\Gamma
1207: _{n}^{\prime}\right) \Gamma_{n}^{\prime\prime\dag}\right\Vert _{\infty}%
1208: ^{4}\mathbb{E}\left\Vert \left( \Gamma^{\prime\prime}-\Gamma_{n}%
1209: ^{\prime\prime}\right) \Gamma^{\prime\prime\dag}\right\Vert _{\infty}%
1210: ^{4}\right) ^{1/2}\\
1211: & \leq M\dfrac{1}{n^{2}\alpha^{4}},
1212: \end{align*}
1213: hence%
1214: \[
1215: \left\Vert \left( \Gamma^{\prime}-\Gamma_{n}^{\prime}\right) \Gamma
1216: _{n}^{\prime\prime\dag}\left( \Gamma^{\prime\prime}-\Gamma_{n}^{\prime\prime
1217: }\right) \Gamma^{\prime\prime\dag}\Gamma^{\prime\ast}\right\Vert =O_{L^{2}%
1218: }\left( \dfrac{1}{\alpha^{2}n}\right) .
1219: \]
1220: The proof of Lemma \ref{cran} is finished.
1221: \end{proof}
1222:
1223: \begin{lemma}
1224: \label{u}We have :%
1225: \[
1226: u_{n}-u=O_{L^{2}}\left( \dfrac{1}{\alpha\sqrt{n}}\right) .
1227: \]
1228:
1229: \end{lemma}
1230:
1231: \begin{proof}
1232: We start with :%
1233: \[
1234: u_{n}-u=\delta_{n}-\delta+\Gamma^{\prime}\Gamma^{\prime\prime\dag}%
1235: \delta^{\prime}-\Gamma_{n}^{\prime}\Gamma_{n}^{\prime\prime\dag}\delta
1236: _{n}^{\prime}.
1237: \]
1238: Clearly $\delta_{n}-\delta=O_{L^{2}}\left( \dfrac{1}{\sqrt{n}}\right) $ and
1239: we study the second term%
1240: \begin{align*}
1241: \Gamma_{n}^{\prime}\Gamma_{n}^{\prime\prime\dag}\delta_{n}^{\prime}%
1242: -\Gamma^{\prime}\Gamma^{\prime\prime\dag}\delta^{\prime} & =\left(
1243: \Gamma_{n}^{\prime}-\Gamma^{\prime}\right) \Gamma_{n}^{\prime\prime\dag
1244: }\delta_{n}^{\prime}+\Gamma^{\prime}\left( \Gamma_{n}^{\prime\prime\dag
1245: }-\Gamma^{\prime\prime\dag}\right) \delta_{n}^{\prime}\\
1246: & +\Gamma^{\prime}\Gamma^{\prime\prime\dag}\left( \delta_{n}^{\prime}%
1247: -\delta^{\prime}\right) .
1248: \end{align*}
1249: Since $\delta_{n}^{\prime}$ is almost surely bounded (see (\ref{asd})),
1250: $\Gamma_{n}^{\prime}-\Gamma=O_{L^{2}}\left( \dfrac{1}{\sqrt{n}}\right) ,$
1251: $\delta_{n}^{\prime}-\delta^{\prime}=O_{L^{2}}\left( \dfrac{1}{\sqrt{n}%
1252: }\right) $ and $\left\Vert \Gamma^{\prime\prime\dag}\right\Vert _{\infty
1253: }=\left\Vert \Gamma_{n}^{\prime\prime\dag}\right\Vert _{\infty}=\alpha^{-1}$
1254: we get :%
1255: \begin{align*}
1256: \left\Vert \left( \Gamma_{n}^{\prime}-\Gamma^{\prime}\right) \Gamma
1257: _{n}^{\prime\prime\dag}\delta_{n}^{\prime}\right\Vert _{W} & =O_{L^{2}%
1258: }\left( \dfrac{1}{\alpha\sqrt{n}}\right) ,\\
1259: \left\Vert \Gamma^{\prime}\Gamma^{\prime\prime\dag}\left( \delta_{n}^{\prime
1260: }-\delta^{\prime}\right) \right\Vert _{W} & =O_{L^{2}}\left( \dfrac
1261: {1}{\alpha\sqrt{n}}\right) .
1262: \end{align*}
1263: The remaining term is%
1264: \begin{align*}
1265: \Gamma^{\prime}\left( \Gamma_{n}^{\prime\prime\dag}-\Gamma^{\prime\prime\dag
1266: }\right) \delta_{n}^{\prime} & =\Gamma^{\prime}\Gamma^{\prime\prime\dag
1267: }\left( \Gamma^{\prime\prime}-\Gamma_{n}^{\prime\prime}\right) \Gamma
1268: _{n}^{\prime\prime\dag}\delta_{n}^{\prime}\\
1269: & =\Gamma^{\prime}\Gamma^{\prime\prime\dag}\left( \Gamma^{\prime\prime
1270: }-\Gamma_{n}^{\prime\prime}\right) \Gamma_{n}^{\prime\prime\dag}\left(
1271: \Gamma_{n}^{\prime\ast}\phi+\Gamma_{n}^{\prime\prime}\psi+u_{n}^{\prime
1272: }\right) ,\\
1273: & =\Gamma^{\prime}\left( \Gamma^{\prime\prime\dag}\right) ^{1/2}\left(
1274: m_{1}+m_{2}+m_{3}\right)
1275: \end{align*}
1276: where
1277: \begin{align*}
1278: m_{1} & =\left( \Gamma^{\prime\prime\dag}\right) ^{1/2}\left(
1279: \Gamma^{\prime\prime}-\Gamma_{n}^{\prime\prime}\right) \left( \Gamma
1280: _{n}^{\prime\prime\dag}\right) ^{1/2}\left( \Gamma_{n}^{\prime\prime\dag
1281: }\right) ^{1/2}\Gamma_{n}^{\prime\ast}\phi,\\
1282: m_{2} & =\left( \Gamma^{\prime\prime\dag}\right) ^{1/2}\left(
1283: \Gamma^{\prime\prime}-\Gamma_{n}^{\prime\prime}\right) \Gamma_{n}%
1284: ^{\prime\prime\dag}\Gamma_{n}^{\prime\prime}\psi,\\
1285: m_{3} & =\left( \Gamma^{\prime\prime\dag}\right) ^{1/2}\left(
1286: \Gamma^{\prime\prime}-\Gamma_{n}^{\prime\prime}\right) \Gamma_{n}%
1287: ^{\prime\prime\dag}e_{n}^{\prime}.
1288: \end{align*}
1289: First we drop $\Gamma^{\prime}\left( \Gamma^{\prime\prime\dag}\right)
1290: ^{1/2}$ since the norm of this operator may be bounded by a constant
1291: independent from $\alpha$ (see Proposition \ref{lodge}). We turn to :%
1292: \begin{align*}
1293: \left\Vert m_{1}\right\Vert & \leq M\left\Vert \left( \Gamma^{\prime\prime
1294: }-\Gamma_{n}^{\prime\prime}\right) \right\Vert _{\infty}\left\Vert \left(
1295: \Gamma_{n}^{\prime\prime\dag}\right) ^{1/2}\right\Vert _{\infty}\left\Vert
1296: \left( \Gamma^{\prime\prime\dag}\right) ^{1/2}\right\Vert _{\infty},\\
1297: \left\Vert m_{2}\right\Vert & \leq\left\Vert \psi\right\Vert _{L}\left\Vert
1298: \left( \Gamma^{\prime\prime\dag}\right) ^{1/2}\right\Vert _{\infty
1299: }\left\Vert \left( \Gamma^{\prime\prime}-\Gamma_{n}^{\prime\prime}\right)
1300: \right\Vert _{\infty}%
1301: \end{align*}
1302: since $\left\Vert \Gamma_{n}^{\prime\prime\dag}\Gamma_{n}^{\prime\prime
1303: }\right\Vert _{\infty}\leq1$ almost surely. The consequence of the display
1304: above is $\left\Vert m_{1}\right\Vert _{L^{2}}=O\left( \dfrac{1}{\alpha
1305: \sqrt{n}}\right) $ and $\left\Vert m_{2}\right\Vert _{L^{2}}=O\left(
1306: \dfrac{1}{\sqrt{\alpha n}}\right) .$
1307:
1308: We can deal with $m_{3}$ as was done within the proof of the preceding Lemma
1309: \ref{cran}. Clearly we may cope with $m_{3}$ as if the random $\Gamma
1310: _{n}^{\prime\prime\dag}$ was replaced by the non random $\Gamma^{\prime
1311: \prime\dag}$. We should study%
1312: \[
1313: \left[ \left( \Gamma^{\prime\prime\dag}\right) ^{1/2}\left( \Gamma
1314: ^{\prime\prime}-\Gamma_{n}^{\prime\prime}\right) \left( \Gamma^{\prime
1315: \prime\dag}\right) ^{1/2}\right] \left[ \left( \Gamma^{\prime\prime\dag
1316: }\right) ^{1/2}e_{n}^{\prime}\right] .
1317: \]
1318: It is enough to get a rate of decrease for each of the these terms. Once again
1319: we have :%
1320: \begin{align*}
1321: \left\Vert \left( \Gamma^{\prime\prime\dag}\right) ^{1/2}\left(
1322: \Gamma^{\prime\prime}-\Gamma_{n}^{\prime\prime}\right) \left( \Gamma
1323: ^{\prime\prime\dag}\right) ^{1/2}\right\Vert _{L^{2}} & =O\left( \frac
1324: {1}{\alpha\sqrt{n}}\right) \\
1325: \left\Vert \left( \Gamma^{\prime\prime\dag}\right) ^{1/2}e_{n}^{\prime
1326: }\right\Vert _{L^{2}} & =O\left( \frac{1}{\sqrt{\alpha n}}\right)
1327: \end{align*}
1328: which completes the proof of Lemma \ref{u}.
1329: \end{proof}
1330:
1331: Now we are ready to go back to (\ref{T1}) and (\ref{T2}) as announced sooner.
1332:
1333: \begin{lemma}
1334: \label{A_n}We have :%
1335: \begin{align*}
1336: A_{n} & =O_{L^{2}}\left( \frac{1}{\alpha\beta\sqrt{n}}\right) ,\\
1337: B_{n} & =O_{L^{2}}\left( \frac{1}{\alpha\beta\sqrt{n}}\right) .
1338: \end{align*}
1339:
1340: \end{lemma}
1341:
1342: \begin{proof}
1343: Since
1344: \[
1345: \left\Vert A_{n}\right\Vert \leq\left\Vert \left( S_{n}+\beta I\right)
1346: ^{-1}\right\Vert _{\infty}\left\Vert u_{n}-u\right\Vert _{W}%
1347: \]
1348: by Lemma \ref{u} and Proposition \ref{KT} we get the first desired result
1349: \end{proof}
1350:
1351: Once again the proof of the second relies on Proposition \ref{KT} and Lemma
1352: \ref{cran}. Indeed%
1353: \begin{align*}
1354: \left\Vert B_{n}\right\Vert _{W} & \leq\left\Vert \left( S_{n}+\beta
1355: I\right) ^{-1}\right\Vert _{\infty}\left\Vert S-S_{n}\right\Vert _{\infty
1356: }\left\Vert \phi\right\Vert _{W}\\
1357: & \leq\dfrac{\left\Vert \phi\right\Vert _{W}}{\beta}\left\Vert S-S_{n}%
1358: \right\Vert _{\infty}%
1359: \end{align*}
1360: hence the result.
1361:
1362: We should deal with the last term. In a first step we prove that $S_{n}$ may
1363: be replaced by $S.$
1364:
1365: \begin{lemma}
1366: \label{C_n}When $\alpha\beta\sqrt{n}\rightarrow+\infty$,%
1367: \[
1368: C_{n}=\beta\left( S+\beta I\right) ^{-1}\phi\left( 1+o\left( 1\right)
1369: \right) .
1370: \]
1371:
1372: \end{lemma}
1373:
1374: \begin{remark}
1375: The preceding equality should be understood with respect to the $L^{2}$ norm.
1376: \end{remark}
1377:
1378: \begin{proof}
1379: Successively,%
1380: \begin{align*}
1381: C_{n} & =\beta\left( S_{n}+\beta I\right) ^{-1}\phi\\
1382: & =\beta\left( \left( S_{n}+\beta I\right) ^{-1}-\left( S+\beta I\right)
1383: ^{-1}\right) \phi+\beta\left( S+\beta I\right) ^{-1}\phi\\
1384: & =\left[ \left( \left( S_{n}+\beta I\right) ^{-1}\left( S-S_{n}\right)
1385: \right) +I\right] \beta\left( S+\beta I\right) ^{-1}\phi
1386: \end{align*}
1387: and
1388: \[
1389: \left\Vert C_{n}\right\Vert \leq\left\Vert \beta\left( S+\beta I\right)
1390: ^{-1}\phi\right\Vert _{W}\left( 1+\left\Vert \left( S_{n}+\beta I\right)
1391: ^{-1}\left( S-S_{n}\right) \right\Vert _{\infty}\right) .
1392: \]
1393: Now it suffices to apply Lemma \ref{A_n} to get the desired result.
1394: \end{proof}
1395:
1396: The next Lemma may be hard to understand at first glance. Within the
1397: forthcoming proof of Theorem \ref{conv} the bias term $C_{n}$ will slightly
1398: change. We refer to displays (\ref{E1}) and (\ref{E2}) below for a deeper understanding.
1399:
1400: \begin{lemma}
1401: \label{placebo}The following holds :%
1402: \[
1403: \left\Vert \Gamma^{1/2}\left( S+\beta I\right) ^{-1}\Gamma^{1/2}\right\Vert
1404: _{\infty}=O\left( \frac{1}{\alpha}\right) .
1405: \]
1406:
1407: \end{lemma}
1408:
1409: \begin{proof}
1410: Once again it takes two steps to get the result. First note that $\Gamma
1411: ^{1/2}\left( S\right) ^{-1}\Gamma^{1/2}$ is a bounded linear operator.
1412: Indeed%
1413: \begin{equation}
1414: S=\Gamma-\Gamma^{\prime}\Gamma^{\prime\prime\dag}\Gamma^{\prime\ast}%
1415: =\Gamma^{1/2}\Lambda_{\alpha}\Gamma^{1/2} \label{queer}%
1416: \end{equation}
1417: where $R=D\Gamma^{1/2},$%
1418: \[
1419: \Lambda_{\alpha}=I-R^{\ast}\left( RR^{\ast}+\alpha I\right) ^{-1}R.
1420: \]
1421: The Schmidt decomposition of $R$ is (see (\ref{mano}) above for the empirical
1422: version) :%
1423: \[
1424: R=\sum_{k\in\mathbb{N}}\mu_{k}\left( u_{k}\otimes v_{k}\right) .
1425: \]
1426: where $\left( u_{k}\right) _{k\in\mathbb{N}}$ (resp. $\left( v_{k}\right)
1427: _{k\in\mathbb{N}}$) is a complete orthonormal system in $W$ (resp. $L$). Hence
1428: :%
1429: \begin{align*}
1430: \Lambda_{\alpha} & =\sum_{k\in\mathbb{N}}\left( 1-\dfrac{\mu_{k}^{2}}%
1431: {\mu_{k}^{2}+\alpha}\right) \left( u_{k}\otimes u_{k}\right) \\
1432: & =\sum_{k\in\mathbb{N}}\dfrac{\alpha}{\mu_{k}^{2}+\alpha}\left(
1433: u_{k}\otimes u_{k}\right) .
1434: \end{align*}
1435: The operator $\Lambda_{\alpha}$ has a bounded inverse%
1436: \[
1437: \Lambda_{\alpha}^{-1}=\dfrac{1}{\alpha}\sum_{k\in\mathbb{N}}\left( \mu
1438: _{k}^{2}+\alpha\right) \left( u_{k}\otimes u_{k}\right)
1439: \]
1440: and $\left\Vert \Lambda_{\alpha}^{-1}\right\Vert _{\infty}=1+\left( \sup
1441: \mu_{k}^{2}\right) /\alpha\leq M/\alpha$ for $M$ large enough (or $\alpha$
1442: small enough).\newline Hence%
1443: \begin{equation}
1444: \Gamma^{1/2}\left( S\right) ^{-1}\Gamma^{1/2}=\Gamma^{1/2}\Gamma
1445: ^{-1/2}\Lambda_{\alpha}^{-1}\Gamma^{-1/2}\Gamma^{1/2}=\Lambda_{\alpha}^{-1}.
1446: \label{BH}%
1447: \end{equation}
1448: Now (second step) we prove that :%
1449: \[
1450: \Gamma^{1/2}\left( S+\beta I\right) ^{-1}\Gamma^{1/2}\ll\Gamma^{1/2}%
1451: S^{-1}\Gamma^{1/2}.
1452: \]
1453: Let us pick a given $x$ in $W$, then
1454: \[
1455: \left\langle \Gamma^{1/2}\left( S+\beta I\right) ^{-1}\Gamma^{1/2}%
1456: x,x\right\rangle _{W}=\left\langle \left( S+\beta I\right) ^{-1}\Gamma
1457: ^{1/2}x,\Gamma^{1/2}x\right\rangle _{W}%
1458: \]
1459: It suffices to get for all $y$ in in the domain of operator $\Gamma^{-1/2}$ :%
1460: \begin{equation}
1461: \left\langle \left( S+\beta I\right) ^{-1}y,y\right\rangle _{W}%
1462: \leq\left\langle S^{-1}y,y\right\rangle _{W} \label{CT}%
1463: \end{equation}
1464: Standard results on the spectrum of $\left( S+\beta I\right) ^{-1}S$ prove
1465: that $\left( S+\beta I\right) ^{-1}S\geq0$ and that $\left\Vert \left(
1466: S+\beta I\right) ^{-1}S\right\Vert \leq1$ which is enough to claim
1467: (\ref{CT}).\newline We are now in position to finixh the proof of the Lemma.
1468: It is plain from (\ref{CT}) that%
1469: \[
1470: \left\Vert \Gamma^{1/2}\left( S+\beta I\right) ^{-1}\Gamma^{1/2}\right\Vert
1471: _{\infty}\leq\left\Vert \Gamma^{1/2}\left( S\right) ^{-1}\Gamma
1472: ^{1/2}\right\Vert _{\infty}=\left\Vert \Lambda_{\alpha}^{-1}\right\Vert
1473: _{\infty}\leq\frac{C}{\alpha}%
1474: \]
1475: which is the claimed result.\bigskip
1476: \end{proof}
1477:
1478: \textbf{Proof of Theorem }\ref{conv}\textbf{:}
1479:
1480: Now starting from (\ref{decomp}) we get%
1481: \begin{align}
1482: \left\Vert \Gamma^{1/2}\left( \phi-\widehat{\phi}\right) \right\Vert
1483: _{W}^{2} & \leq M\left\Vert \Gamma^{1/2}\left( A_{n}+B_{n}+C_{n}\right)
1484: \right\Vert _{W}^{2}\nonumber\\
1485: & \leq M\left( \left\Vert A_{n}\right\Vert _{W}^{2}+\left\Vert
1486: B_{n}\right\Vert _{W}^{2}+\left\Vert \Gamma^{1/2}C_{n}\right\Vert _{W}%
1487: ^{2}\right) . \label{E1}%
1488: \end{align}
1489:
1490:
1491: Lemmas \ref{A_n} gives the rates of convergence for $\left\Vert A_{n}%
1492: \right\Vert _{W}^{2}$ and $\left\Vert B_{n}\right\Vert _{W}^{2}$ respectively.
1493: But Lemma \ref{C_n} is unfortunately not enough to get a rate in the last
1494: term. However this previous Lemma enables to focus on :%
1495: \begin{equation}
1496: \beta\Gamma^{1/2}\left( S+\beta I\right) ^{-1}\phi=\beta\Gamma^{1/2}\left(
1497: S+\beta I\right) ^{-1}\Gamma^{1/2}\Gamma^{-1/2}\phi\label{E2}%
1498: \end{equation}
1499: and
1500: \begin{equation}
1501: \left\Vert \Gamma^{1/2}C_{n}\right\Vert _{W}^{2}\leq M\beta^{2}\left\Vert
1502: \Gamma^{1/2}\left( S+\beta I\right) ^{-1}\Gamma^{1/2}\right\Vert _{\infty
1503: }^{2}\left\Vert \Gamma^{-1/2}\phi\right\Vert _{W}. \label{marin}%
1504: \end{equation}
1505: By assumption $\mathbf{A4}$, $\left\Vert \Gamma^{-1/2}\phi\right\Vert _{W}$ is
1506: finite. We deal with the central term, namely :%
1507: \begin{align*}
1508: \Gamma^{1/2}\left( S+\beta I\right) ^{-1}\Gamma^{1/2} & =\Gamma
1509: ^{1/2}\left( \Gamma^{1/2}\Lambda_{\alpha}\Gamma^{1/2}+\beta I\right)
1510: ^{-1}\Gamma^{1/2}\\
1511: & \ll\Gamma^{1/2}\left( \Gamma^{1/2}\Lambda_{\alpha}\Gamma^{1/2}\right)
1512: ^{-1}\Gamma^{1/2}=\Lambda_{\alpha}^{-1}.
1513: \end{align*}
1514: (see (\ref{queer})) and%
1515: \[
1516: \left\Vert \Lambda_{\alpha}^{-1}\right\Vert _{\infty}^{2}=O\left( \alpha
1517: ^{-2}\right) .
1518: \]
1519: Collecting this last display with (\ref{marin}) we get%
1520: \[
1521: \left\Vert \Gamma^{1/2}C_{n}\right\Vert _{W}^{2}=O\left( \frac{\beta^{2}%
1522: }{\alpha^{2}}\right) .
1523: \]
1524: This finishes the proof of Theorem \ref{conv}.
1525:
1526: \begin{thebibliography}{99} %
1527:
1528:
1529: \bibitem {AF}Adams R.A. and Fournier J.J.F., 2003. Sobolev spaces, Academic
1530: Press, 2nd ed.
1531:
1532: \bibitem {Amato.et.al.CSDA}Amato U., Antoniadis A., Feiss I., 2006. Dimension
1533: reduction in functional regression with applications, to appear in
1534: \textit{Computational Statistics and Data Analysis.}
1535:
1536: \bibitem {Bos3}Bosq, D., 2000. \textit{Linear processes in function spaces}.
1537: Lectures notes in statistics. Springer Verlag.
1538:
1539: \bibitem {Brown.Fearn.Vanucci}Brown, P.J., Fearn, T. and Vanucci M., 2001.
1540: Bayesian wavelet regression on curves with application to a spectroscopic
1541: calibration problem, \textit{Journal of the American Statistical
1542: Association.}, 96 (454), 398--408.
1543:
1544: \bibitem {CH}Cai T., Hall, P., 2006. Prediction in functional linear
1545: regression, \textit{Annals of Statistics}, \textbf{34}, n$%
1546: %TCIMACRO{\U{b0}}%
1547: %BeginExpansion
1548: {{}^\circ}%
1549: %EndExpansion
1550: 5$.
1551:
1552: \bibitem {Cardot.SEFL}Cardot H., Ferraty F., Sarda P., 2006. Spline estimators
1553: for the functional linear model: Consistency, Application and Splus
1554: implementation, To appear in \textit{Statistica Sinica}.
1555:
1556: \bibitem {CMS2006}Cardot H., Mas A., Sarda P, 2006. Weak convergence in the
1557: functional linear model. To appear in \textit{Probability Theory and Related
1558: Fields}.
1559:
1560: \bibitem {DS}Dunford, N. and Schwartz, J.T., 1988. \textit{Linear Operators,
1561: Vol. I \& II}. Wiley Classics Library.
1562:
1563: \bibitem {FV}Ferraty, F., Vieu P., 2003. The functional nonparametric model
1564: and application to spectrometric data. \textit{Computational Statistics}
1565: \textbf{17} n$%
1566: %TCIMACRO{\U{b0}}%
1567: %BeginExpansion
1568: {{}^\circ}%
1569: %EndExpansion
1570: 4,$ 545-564.
1571:
1572: \bibitem {FV2}Ferraty, F., Vieu P., 2006. Nonparametic Functional Data
1573: Analysis, Springer.
1574:
1575: \bibitem {Frank.Friedman}Frank I.E. and Friedman J.H., 1993. A statistical
1576: view of some chemometrics regression tools, \textit{Technometrics},
1577: \textbf{35}, no 2, 109-135.
1578:
1579: \bibitem {GGK}Gohberg, I., Goldberg, S. and Kaashoek,M.A., 1991.
1580: \textit{Classes of linear operators Vol I \& II. Operator Theory : advances
1581: and applications}, Birkha\"{u}ser Verlag.
1582:
1583: \bibitem {Hastie.Mallows}Hastie T. and Mallows C., 1993. Discussion on the
1584: paper of Frank I.E. and Friedman J.H., 1993, A statistical view of some
1585: chemometrics regression tools, \textit{Technometrics}, \textbf{35}, no 2, 140-143.
1586:
1587: \bibitem {LT}Ledoux, M. and Talagrand, M, 1991 : \textit{Probability in Banach
1588: spaces. Isoperimetry and processes}. Ergebnisse der Mathematik und ihrer
1589: Grenzgebiete, \textbf{23}. Springer-Verlag, Berlin.
1590:
1591: \bibitem {Marx.Eilers}Marx, B.D. and Eilers, P.H.C. (2002) Multivariate
1592: calibration stability: a comparison of methods, \textit{Journal of
1593: Chemometrics}, \textbf{16}, 129-140.
1594:
1595: \bibitem {MP}Mas A, Pumo, B, 2006. The ARHD model. To appear in JSPI.
1596: Technical report available at
1597: \textit{http://fr.arxiv.org/PS\_cache/math/pdf/0502/0502285.pdf}
1598:
1599: \bibitem {marion.pumo}Marion J.M., Pumo B., 2004. Comparaison des mod\`{e}les
1600: ARH(1) et ARHD(1) sur des donn\'{e}es physiologiques, \textit{Annales de
1601: l'ISUP}, \textbf{48}, 3, pp. 29-38.
1602:
1603: \bibitem {Osborne.et.al.}Osborne, B.J., Fearn, T., Miller, A.R. and Douglas,
1604: S. (1984) Application of Near-Infrared Reflectance Spectroscopy to
1605: Compositional Analysis of Biscuits Dougts, \textit{J. of the Sc. of Food and
1606: Agricult.}, \textbf{35}, 99--105
1607:
1608: \bibitem {Ram1}Ramsay J.O., 2000. Differential equation models for statistical
1609: functions, \textit{Canadian Journal of Statistics}, \textbf{28}, n$%
1610: %TCIMACRO{\U{b0}}%
1611: %BeginExpansion
1612: {{}^\circ}%
1613: %EndExpansion
1614: 2,$ 225-240.
1615:
1616: \bibitem {RamDal}Ramsay J.O., Dalzell C.J., 1991, Some tools for functional
1617: data analysis (with dsicussion), \textit{Journal of the Royal Statistical
1618: Society}, B, \textbf{53}, 539-572.
1619:
1620: \bibitem {ramsey}Ramsay J.O., Silverman B.W., 1997. \textit{Functional Data
1621: Analysis}, Springer.
1622:
1623: \bibitem {ramsay2}Ramsay J.O., Silverman B.W., 2002. \textit{Applied
1624: Functional Data Analysis: Methods and Case Studies}, Springer.
1625:
1626: \bibitem {Silv}Silverman B.W., 1996. Smoothed functional principal component
1627: analysis by choice of norm, \textit{Annals of Statistics}, \textbf{24}, No 1, 1-24.
1628:
1629: \bibitem {TA}Tikhonov A.N., Arsenin V.Y., 1977. \textit{Solutions of ill-posed
1630: problems}. V.H. Winstons and sons, Washington..
1631:
1632: \bibitem {Z}Ziemer W.P., 1989. Weakly differentiable functions.
1633: \textit{Sobolev spaces and functions of bounded variations}. Graduate Text in
1634: Mathematics 120. Springer-Verlag, New-York.
1635: \end{thebibliography}
1636:
1637:
1638: \end{document}