8f144654e9968a5d.tex
1: \begin{definition}
2: \label{DefEntropy}
3: $ \Phi = \left\{\phi_{d_{VC},c} : I\times H \longrightarrow \mathbb{R} \right\}$ where $I\in [a,b]\subset \mathbb{R}^{+}$, $d_{VC}\in \left(1, M\right]\subset \mathbb{N}$. For the purpose of our proof, initially we fix $c$ so $\phi = \phi(d_{VC})$.  For $s = 0, 1, 2, \cdots,$ let $\{\phi_{j}^{s}\}_{j=1}^{N_{s}}$ be the minimal $2^{-s}R$-covering set of $\left(2^{-s}R,\Phi, \parallel \cdot \parallel_{Q}\right)$ for given $s\in I$ and $R> 0$. Now, $N_{s} = N\left(2^{-s}R, \Phi, Q\right)$ is the smallest number of balls of radius $2^{-s}R$ that cover $\Phi$.  For each $d_{VC}$, there exists a $\phi_{j}^{s} \in \left\{\phi_{1}^{s}, \phi_{2}^{s}, \cdots, \phi_{N_{s}}^{s}\right\}$ such that $\parallel\phi_{d_{VC}} - \phi_{j}^{s} \parallel_{Q} \leq 2^{-s}R$. Because $\Phi$ is compact we choose $R>0$ so that $\sup_{d_{VC}}\parallel \phi_{d_{VC}}\parallel_{Q}\leq R$.
4: Now consider the class $\Phi(R) = \left\{\phi \in \Phi : \parallel \phi_{d_{VC}}-\phi_{d_{VC}}\parallel_{Q} \leq R\right\}$, i.e. the elements of $\Phi$ within $R$ of $\phi_{d_{VC}}$. obviously $j = j(d_{VC})$ and as derived in Theorem \ref{Theo23and4}, let
5: \begin{equation}
6:  \label{phibound}
7:  \phi_{d_{VC}}\left(n\right) = c\sqrt{\frac{d_{VC}}{n}\log\left(\frac{2ne}{d_{VC}}\right)}.
8: \end{equation}
9: \end{definition}
10: