1: \begin{abstract}

2: Block coordinate descent methods and stochastic subgradient methods

3: have been extensively studied in optimization and machine learning.

4: By combining randomized block sampling with stochastic subgradient

5: methods based on dual averaging (\cite{nesterov2009primal,Xiao:2010:DAM}),

6: we present stochastic block dual averaging (SBDA)---a novel class

7: of block subgradient methods for convex nonsmooth and stochastic optimization.

8: SBDA requires only a block of subgradients and updates blocks of variables

9: and hence has significantly lower iteration cost than traditional

10: subgradient methods. We show that the SBDA-based methods exhibit the

11: optimal convergence rate for convex nonsmooth stochastic optimization.

12: More importantly, we introduce randomized stepsize rules and block

13: sampling schemes that are adaptive to the block structures, which

14: significantly improves the convergence rate w.r.t. the problem parameters.

15: This is in sharp contrast to recent block subgradient methods applied

16: to nonsmooth deterministic or stochastic optimization (\cite{Dang:arXiv1309.2249,nesterov2014subgradient}).

17: For strongly convex objectives, we propose a new averaging scheme

18: to make the regularized dual averaging method optimal, without having

19: to resort to any accelerated schemes.

20: \end{abstract}

21: