03b3cc0311b1bd74.tex
1: \begin{abstract}
2: In this work, we introduce a new framework for unifying and systematizing the performance analysis of first-order black-box optimization algorithms for unconstrained convex minimization. % over finite-dimensional Euclidean spaces.
3: The low-cost iteration complexity enjoyed by first-order  algorithms renders them particularly relevant for applications in machine learning and large-scale data analysis. % \red{however}, existing proofs of convergence 
4: %of such optimization algorithms 
5: %consist mostly of  case-by-case analyses.
6: %On the other hand, 
7: Our approach is based on sum-of-squares optimization, which allows to    introduce a hierarchy of semidefinite programs (SDPs) that  give increasingly better   convergence bounds for higher levels of the hierarchy. 
8: %as the eve  that are increasingly  better for higher levels,.
9:  %(SDPs), for which higher levels of the hierarchy lead to improved bounds. 
10: The  (dual of the) first level of the sum-of-squares hierarchy corresponds to the SDP reformulation of the Performance Estimation Problem, first  
11: % an area of study 
12:  introduced  by Drori and Teboulle  [{\em Math.\ Program.}, 145(1):451--482, 2014] and developed further by  Taylor, Hendrickx, and Glineur [{\em Math.\ Program.}, 161(1):307--345, 2017].
13: %\cite{Drori2014,Taylor}.}
14: Illustrating the usefulness of our approach, we recover, in a unified manner,   several known convergence bounds for four widely-used first-order algorithms, 
15: %in a unified manner, 
16: and also derive new convergence result{s} for noisy gradient descent with   inexact line search methods.
17: \end{abstract}
18: