1: \begin{abstract}

2: The alternating direction method of multipliers (ADMM) is

3: a powerful optimization solver 

4: in machine learning.

5: Recently, stochastic ADMM has been integrated with variance reduction methods for stochastic

6: gradient, leading to SAG-ADMM and SDCA-ADMM that have fast convergence rates

7: and low iteration complexities.

8: However, their space requirements can still be high. 

9: In this paper, we propose an integration of 

10: ADMM with

11: the method of stochastic variance reduced gradient (SVRG).

12: Unlike another recent integration attempt called

13: SCAS-ADMM,

14: the proposed algorithm retains the fast convergence benefits of 

15: SAG-ADMM and SDCA-ADMM,

16: but is more advantageous in that 

17: its storage requirement

18: is very low,  even independent of the sample size $n$. We also extend the proposed method for nonconvex problems, and obtain a convergence rate of $O(1/T)$.

19: Experimental results demonstrate that it is as fast as SAG-ADMM and SDCA-ADMM, much faster than

20: SCAS-ADMM, and can be used on much bigger data sets.

21: \end{abstract}

22: