1: \begin{abstract}

2: In this paper, we generalize the well-known

3: Nesterov's accelerated gradient (AG) method, originally designed for

4: convex smooth optimization, to solve

5: nonconvex and possibly stochastic optimization problems.

6: We demonstrate that by properly specifying the stepsize policy,

7: the AG method exhibits the best known rate of convergence for solving general

8: nonconvex smooth optimization problems by using first-order information,

9: similarly to the gradient descent method. We then consider

10: an important class of composite optimization problems

11: and show that the AG method can solve

12: them uniformly, i.e., by using the same aggressive stepsize policy

13: as in the convex case, even if the problem turns out to be nonconvex.

14: We demonstrate that the AG method exhibits an optimal rate of convergence

15: if the composite problem is convex, and improves

16: the best known rate of convergence if the problem

17: is nonconvex. Based on the AG method,

18: we also present new nonconvex stochastic approximation

19: methods and show that they can improve a few existing rates of convergence

20: for nonconvex stochastic optimization. To the best of our knowledge, this is the first time that the

21: convergence of the AG method has been established for

22: solving nonconvex nonlinear  programming

23: in the literature.

24: 

25: \vspace{.1in}

26: 

27: \noindent {\bf Keywords: nonconvex optimization, stochastic programming, accelerated gradient, complexity}

28: 

29: \noindent {\bf AMS 2000 subject classification:} 62L20, 90C25, 90C15, 68Q25,

30: 

31: \end{abstract}

32: