c2bddb73f40d91fc.tex
1: \begin{abstract}
2: We consider gradient descent with constant stepsizes and derive exact worst-case convergence rates on the minimum gradient norm of the iterates. Our analysis covers all possible stepsizes and arbitrary upper/lower bounds on the curvature of the objective function, thus including convex, strongly convex and weakly convex (hypoconvex) objective functions. 
3: 
4: Unlike prior approaches, which rely solely on inequalities connecting consecutive iterations, our analysis employs inequalities involving an iterate and its two predecessors. While this complicates the proofs to some extent, it enables us to achieve, for the first time, an exact full-range analysis of gradient descent for any constant stepsizes (covering, in particular, normalized stepsizes greater than one), whereas the literature contained only conjectured rates of this type. 
5: 
6: % Our analysis accommodates arbitrary bounds on both the upper and lower curvatures of the smooth objective function. 
7: In the nonconvex case, allowing arbitrary bounds on upper and lower curvatures extends existing partial results that are valid only for gradient Lipschitz functions (i.e., where lower and upper bounds on curvature are equal), leading to improved rates for weakly convex functions. 
8: 
9: From our exact rates, we deduce the optimal constant stepsize for gradient descent. Leveraging our analysis, we also introduce a new variant of gradient descent based on a unique, fixed sequence of variable stepsizes, demonstrating its superiority over any constant stepsize schedule.% in the (strongly) convex case.
10: \keywords{Gradient descent \and Performance estimation \and Nonconvex functions \and Convex functions}
11: % \PACS{PACS code1 \and PACS code2 \and more}
12: %\subclass{MSC code1 \and MSC code2 \and more}
13: \end{abstract}
14: