1: \begin{abstract}

2: %

3: In stochastic optimization,

4: the population risk is generally approximated by the empirical risk.

5: However, in the large-scale setting, minimization of the empirical risk may be 

6: computationally restrictive.

7: In this paper, we design

8: an efficient algorithm to approximate the population risk minimizer

9: in generalized linear problems such as

10: binary classification with surrogate losses and generalized linear regression models.

11: We focus on large-scale problems,

12: where the iterative minimization of the empirical risk is computationally intractable, i.e.,

13: the number of observations $n$

14: is much larger than the dimension of the parameter $p$, i.e. $n \gg p \gg 1$.

15: %

16: We show that under random sub-Gaussian design,

17: the true minimizer of the population risk is approximately proportional to the

18: corresponding ordinary least squares (OLS) estimator.

19: Using this relation,

20: we design an algorithm that achieves the same accuracy as 

21: the empirical risk minimizer

22: through iterations that 

23: attain up to a cubic convergence rate,

24: and that are cheaper than 

25: any batch optimization algorithm by at least a factor of $\mathcal{O}(p)$.

26: %

27: We provide theoretical guarantees for our algorithm,

28: and analyze the convergence behavior in terms of data dimensions.

29: %

30: Finally, we demonstrate the performance of 

31: our algorithm on well-known classification and regression problems,

32: through extensive numerical studies 

33: on large-scale datasets, and show that

34: it achieves the highest performance compared to several other widely

35: used and specialized optimization algorithms.

36: %%

37: \end{abstract}

38: