1: \begin{abstract}

2: In this paper, we study the finite-sum convex optimization problem

3: focusing on the general convex case. Recently, the study of variance

4: reduced (VR) methods and their accelerated variants has made exciting

5: progress. However, the step size used in the existing VR algorithms

6: typically depends on the smoothness parameter, which is often unknown

7: and requires tuning in practice. To address this problem, we propose

8: two novel adaptive VR algorithms: \textit{Adaptive Variance Reduced

9: Accelerated Extra-Gradient} (AdaVRAE) and\textit{ Adaptive Variance

10: Reduced Accelerated Gradient} (AdaVRAG). Our algorithms do not require

11: knowledge of the smoothness parameter. AdaVRAE uses $\mathcal{O}\left(n\log\log n+\sqrt{\frac{n\beta}{\epsilon}}\right)$

12: gradient evaluations and AdaVRAG uses $\mathcal{O}\left(n\log\log n+\sqrt{\frac{n\beta\log\beta}{\epsilon}}\right)$

13: gradient evaluations to attain an $\mathcal{O}(\epsilon)$-suboptimal

14: solution, where $n$ is the number of functions in the finite sum

15: and $\beta$ is the smoothness parameter. This result matches the

16: best-known convergence rate of non-adaptive VR methods and it improves

17: upon the convergence of the state of the art adaptive VR method, AdaSVRG.

18: We demonstrate the superior performance of our algorithms compared

19: with previous methods in experiments on real-world datasets.

20: \end{abstract}

21: