1: \begin{abstract}

2: Stochastic gradient algorithms estimate the gradient based on only one or a few samples

3: and enjoy low computational cost per iteration. They have been widely used in large-scale optimization problems.

4: However, stochastic gradient algorithms are usually slow to converge and achieve sub-linear convergence rates,

5: due to the inherent variance in the gradient computation. To accelerate the convergence,

6: some variance-reduced stochastic gradient algorithms, e.g., proximal stochastic variance-reduced gradient (Prox-SVRG) algorithm,

7: have recently been proposed to solve strongly convex problems. Under the strongly convex condition,

8: these variance-reduced stochastic gradient algorithms achieve a linear convergence rate. However,

9: many machine learning problems are convex but not strongly convex. In this paper,

10: we introduce Prox-SVRG and its projected variant called Variance-Reduced Projected Stochastic Gradient (VRPSG) to

11: solve a class of non-strongly convex optimization problems widely used in machine learning. As the main technical contribution of this paper,

12: we show that both VRPSG and Prox-SVRG achieve a linear convergence rate without strong convexity.

13: A key ingredient in our proof is a Semi-Strongly Convex (SSC) inequality which is the first to be rigorously proved for a class of non-strongly convex problems in both constrained

14: and regularized settings. Moreover, the SSC inequality is independent of algorithms and may be applied to analyze other stochastic gradient algorithms

15: besides VRPSG and Prox-SVRG, which may be of independent interest.

16: To the best of our knowledge, this is the first work that establishes the linear convergence rate for the

17: variance-reduced stochastic gradient algorithms on solving both constrained and regularized problems without strong convexity.

18: \end{abstract}

19: