1: \begin{abstract}

2: The alternating direction method of

3: multipliers (ADMM) is now widely used in many fields, and its

4: convergence was proved when two blocks of variables are

5: alternately updated. It is computationally beneficial to extend the ADMM

6: directly to the case of a multi-block convex minimization problem. Unfortunately, such

7: an extension fails to converge even when solving a simple square system of linear equations.

8: In this paper, however, we prove that, if in each step one randomly and independently permutes the updating order of any given number of blocks followed by the regular multiplier update, the method will converge in expectation for solving the square system of linear equations.

9:   Our result indicates that for ADMM the cyclic update rule and the random permutation update rule

10:  are fundamentally different.

11:  To the best of our knowledge, this is the first example that such a difference is rigorously established for a specific optimization algorithm,

12:   although the random permutation rule has been reported to be experimentally better than the cyclic rule 

13:   in many large-scale problems. 

14:   % for other optimization algorithms  for an optimization algorithm

15: Our analysis technique of random permutation might be useful in other contexts. % of independent interest.

16: % for any $n$.    %  very popular in large scale optimization.

17: 

18: % permutation-based

19: \end{abstract}