069b70e64629eda7.tex
1: \begin{abstract}
2:   We modify Nesterov's constant step gradient method for strongly convex
3:   functions with Lipschitz continuous gradient described in Nesterov's
4:   book. Nesterov shows that $f(x_k) - f^* \leq L \prod_{i=1}^k ( 1 - \alpha_k )
5:   \| x_0 - x^* \|_2^2$ with $\alpha_k = \sqrt{\rho}$ for all $k$, where $L$ is
6:   the Lipschitz gradient constant and $\rho$ is the reciprocal condition number
7:   of $f(x)$. Hence the convergence rate is $1-\sqrt{\rho}$. In this work, we try
8:   to accelerate Nesterov's method by adaptively searching for an $\alpha_k >
9:   \sqrt{\rho}$ at each iteration. The proposed method evaluates the gradient
10:   function at most twice per iteration and has some extra Level 1 BLAS
11:   operations. Theoretically, in the worst case, it takes the same number of
12:   iterations as Nesterov's method does but doubles the gradient calls. However,
13:   in practice, the proposed method effectively accelerates the speed of
14:   convergence for many problems including a smoothed basis pursuit denoising
15:   problem.
16: \end{abstract}
17: