6f2485c833044319.tex
1: \begin{abstract}
2: 	Nonlinear conjugate gradients are among the most popular
3: 	techniques for solving continuous optimization problems. Although 
4: 	these schemes have long been studied from a global convergence 
5: 	standpoint, their worst-case complexity properties have yet to be 
6: 	fully understood, especially in the nonconvex setting. In particular, 
7: 	it is unclear whether \revised{nonlinear conjugate gradient} methods possess better 
8: 	guarantees than first-order methods such as gradient descent. \revised{Meanwhile,
9: 	recent experiments} have shown \revised{impressive} performance of 
10: 	standard nonlinear conjugate gradient \revised{techniques} on \revised{certain} 
11: 	nonconvex problems, 
12: 	even when compared with methods endowed with the best known complexity 
13: 	guarantees. 	
14: 	
15: 	In this paper, we propose a nonlinear conjugate gradient \revised{scheme} 
16: 	based on a simple line-search paradigm and a modified restart condition.
17: 	These two ingredients allow for monitoring the 
18: 	properties of the search directions, which is instrumental 
19: 	in obtaining complexity guarantees. Our complexity results illustrate 
20: 	the possible discrepancy between nonlinear conjugate gradient methods 
21: 	and classical gradient descent. \revised{A numerical investigation on 
22: 	nonconvex robust regression problems as well as a standard benchmark 
23: 	illustrate that the restarting 
24: 	condition can track the behavior of a standard implementation.}
25: \end{abstract}