13cfc39fbd26a938.tex
1: \begin{proof}
2:     Let $f_{X^n}$ be the cumulative loss, $f_{X^n}=\sum_{i=1}^nf_{X_i}$, and observe that $f_{X^n}$ is $Ln$-upper smooth and logarithmic.
3:     Let $\Fcal=\mysetinline{f_{X^n}}{X^n\in\Xcal^n}$ and $\gamma(\theta)=\lambda \norm{\theta}_1$.
4:     Also, let $\gamma_0$ be the indicator penalty of the set $\Hcal_B$
5:     such that
6:     $\gamma_0(\theta)=0$ if and only if $\theta\in\Hcal_B$ and otherwise
7:     $\gamma_0(\theta)=\infty$.
8:     Then, we have $\reg_n(w|\Hcal_B)=\lreg(w|\gamma_0)$
9:     where $\lreg$ is taken with respect to $f_{X^n}$.
10:     Now, observe that
11:     \begin{align*}
12:         \lreg(w_{\lambda,Ln}^\st|\gamma_0)
13:         &\le\lreg(w_{\lambda,Ln}^\st|\gamma - \lambda B)
14:         \\&\quad (\because \gamma_0\ge \gamma - \lambda B)
15:         % \\&=\lambda B+\lreg(w_{\lambda,Ln}^\st|\gamma)
16:         \\&\le
17:         \ln w_{\lambda,Ln}^\st\sbr{e^{-\gamma+\lambda B}},
18:         \\&\quad (\because\mathrm{Theorem~\ref{thm:set_repr_envelope}})
19:         \\&=
20:         \lambda B + \ln w_{\lambda,Ln}^\st\sbr{e^{-\gamma}},
21:     \end{align*}
22:     which, combined with Corollary~\ref{cor:ec_ell1} where $\lambda=\sqrt{2Ln\ln (d/\sqrt{Ln})}$,
23:     yields the asymptotic equality.
24:     The proof of the minimaxity is adopted from the existing analysis on the minimax \emph{risk}~(see Section~\ref{sec:proof_minimaxity} for the rigorous proof and Section~\ref{sec:discussion_minimax_risk} for detailed discussions).
25: \end{proof}
26: