25b5d8258c805383.tex
1: \begin{abstract}
2: We consider the problem of minimizing a sum of $n$ functions 
3: via projected iterations onto a convex parameter set $\C \subset \reals^p$ where $n\gg p\gg 1$. 
4: In this regime, algorithms which utilize sub-sampling techniques are known to be effective.
5: In this paper, we use sub-sampling techniques together with 
6: eigenvalue thresholding to design a new randomized batch algorithm 
7: which possesses comparable convergence rate to Newton's method, 
8: yet has much smaller per-iteration cost. 
9: The proposed algorithm is robust in terms of starting point and 
10: step size, and enjoys a composite convergence rate, namely, 
11: quadratic convergence at start and linear convergence 
12: when the iterate is close to the minimizer. 
13: We develop its theoretical analysis which also allows us to 
14: select near-optimal algorithm parameters. 
15: Our theoretical results can be used to obtain 
16: convergence rates of previously proposed sub-sampling based algorithms as well. 
17: We demonstrate how our results apply to well-known machine learning problems.
18: Lastly, we evaluate the performance of our algorithm on several datasets 
19: under various scenarios. 
20: \end{abstract}
21: