1: \begin{abstract}

2: We introduce a new recursive aggregation procedure called Bernstein Online Aggregation (BOA). Its exponential weights include a second order refinement. The procedure is optimal for the model selection problem in the iid setting, see \cite{tsybakov:2003};  the excess of risk of its batch version achieves the fast rate of convergence $\log(M)/n$ in deviation when the loss is Lipschitz and strongly convex. The BOA procedure is the first online algorithm that satisfies this optimal fast rate. The second order refinement  is required to achieve the optimality in deviation as the classical exponential weights cannot be optimal, see \cite{audibert:2009}. This refinement is settled thanks to a new online to batch conversion that estimates the deviations in the stochastic environment with random second order terms. It converts a second order regret bound for BOA similar than in \cite{gaillard:stoltz:vanerven:2014} to a bound on the cumulative predictive risk in a general stochastic context. The empirical second order term is shown to be sufficiently small to assert the fast rate in the iid setting when the loss is Lipschitz and strongly convex. We also introduce a multiple learning rates version of BOA. This fully adaptive BOA procedure is also optimal, up to a $\log\log(n)$ factor. \end{abstract}

3: