1: \begin{abstract}%

2: We introduce a generic scheme for accelerating gradient-based optimization

3: methods in the sense of Nesterov. The approach, called Catalyst, builds upon the inexact

4: accelerated proximal point algorithm for minimizing a convex objective function, and

5: consists of approximately solving a sequence of well-chosen auxiliary problems,

6: leading to faster convergence.  

7: One of the keys to achieve acceleration in theory and in practice is to

8: solve these sub-problems with appropriate accuracy by using the right stopping

9: criterion and the right warm-start strategy. We give practical guidelines to use Catalyst and present a comprehensive analysis of its global complexity. 

10: We show that Catalyst applies to a large class of algorithms,

11: including gradient descent, block coordinate descent, incremental algorithms

12: such as SAG, SAGA, SDCA, SVRG, MISO/Finito, and their proximal variants. For

13: all of these methods, we establish faster rates using the Catalyst acceleration, for

14: strongly convex and 

15: non-strongly convex objectives.  We conclude with extensive experiments showing

16: that acceleration is useful in practice, especially for ill-conditioned

17: problems.

18: \end{abstract}

19: