4f8cd8ad6b82851f.tex
1: \begin{abstract}
2: We show that the exact worst-case performance of fixed-step
3: first-order methods for {unconstrained optimization of} smooth (possibly strongly) convex functions
4: can be obtained by solving convex programs.
5: 
6: Finding the worst-case performance of a black-box first-order method
7: is formulated as an optimization problem over a set of smooth (strongly)
8: convex functions and initial conditions. We develop closed-form necessary and
9: sufficient conditions for smooth (strongly) convex interpolation,
10: which provide a finite representation for those functions. This
11: allows us to reformulate the worst-case performance estimation problem
12: as an equivalent finite dimension-independent semidefinite
13: optimization problem, whose exact solution can be recovered up to numerical
14: precision. Optimal solutions to this performance estimation problem
15: provide both worst-case performance bounds and explicit functions
16: matching them, as our smooth (strongly) convex interpolation procedure is
17: constructive.
18: 
19: Our works build on those of Drori and Teboulle
20: in {[Math.\@ Prog.\@ 145 (1-2), 2014]} who introduced and solved relaxations of the
21: performance estimation problem for smooth convex functions.
22: 
23: We apply our approach to different fixed-step first-order methods with
24: several performance criteria, including objective function accuracy and
25: gradient norm. We conjecture several numerically supported worst-case
26: bounds on the performance of the {fixed-step} gradient, fast gradient and optimized {gradient} methods, both in the smooth convex and the smooth strongly
27: convex cases, and deduce tight estimates of the optimal step size for
28: the gradient method.
29: %\keywords{Smooth convex minimization \and Smooth convex interpolation \and First-order methods \and Worst-case analysis \and Rates of convergence \and Semidefinite programming}
30: %%\PACS{PACS code1 \and PACS code2 \and more}
31: %\subclass{90C25 \and 90C30 \and 90C60 \and 68Q25 \and 90C22 }
32: \end{abstract}
33: