1: \begin{abstract} 

2: We propose a new stochastic dual coordinate ascent technique 

3: that can be applied to a wide range of regularized learning problems.

4: Our method is based on 

5: Alternating Direction Multiplier Method (ADMM) to deal with complex regularization functions

6: such as structured regularizations.

7: Although the original ADMM is a batch method, 

8: the proposed method offers a stochastic update rule where each iteration requires only one or few sample observations.

9: Moreover, our method can naturally afford mini-batch update and it gives speed up of convergence.

10: We show that, under mild assumptions, our method converges exponentially.

11: The numerical experiments show that our method actually performs efficiently.

12: 

13: 

14: \noindent {\bf Keywords}: Stochastic Dual Coordinate Ascent, Alternating Direction Multiplier Method, Exponential Convergence, Structured Sparsity.

15: \end{abstract}