1: \begin{abstract}

2: In this paper, we study large-scale convex optimization algorithms based on the Newton method applied to regularized generalized self-concordant losses, which include logistic regression and softmax regression.  We first prove that our new simple scheme based on a sequence of problems with decreasing regularization parameters is provably globally convergent, that this convergence is linear with a constant factor which scales only logarithmically with the condition number. In the parametric setting, we obtain an algorithm with the same scaling than regular first-order methods but with an improved behavior, in particular in ill-conditioned problems. Second, in the non-parametric machine learning setting, we provide an explicit algorithm combining  the previous scheme with Nystr\"om projection techniques, and prove that it achieves optimal generalization bounds with a time complexity of order $O(n\Deff_\la)$, a memory complexity of order $O(\Deff_\la^2)$ and {\em no dependence on the condition number}, generalizing the results known for least-squares regression. Here $n$ is the number of observations and $\Deff_\la$ is the associated degrees of freedom. In particular, this is the first large-scale algorithm to solve logistic and softmax regressions in the non-parametric setting with large condition numbers and theoretical guarantees.

3: \end{abstract}

4: