1: \begin{abstract}

2: 

3: % 1. Motivation:

4: \rvs{Update order is one of the major design choices of  block decomposition algorithms.}

5: % \rvs{Gaussian back substitution and symmetric Gauss-Seidel are two deterministic symmetrized orders which can be applied in popular methods with block decomposition, such as BCD and ADMM.}

6: \rvs{There are at least two classes

7: of deterministic update orders: nonsymmetric (e.g. cyclic order) and symmetric (e.g. Gaussian back substitution or symmetric Gauss-Seidel)}.

8: \rvs{Recently}, Coordinate Descent (CD) with cyclic order was shown to be $O(n^2)$ times slower than

9: randomized versions in the worst-case. 

10: A natural question arises: can the symmetrized orders achieve faster convergence rates

11: than the cyclic order, or even getting close to the randomized versions? 

12: % 2. Results:

13: In this paper, we give a negative answer to this question. We show that both Gaussian \rvs{b}ack \rvs{s}ubstitution (GBS) and symmetric Gauss-Seidel (sGS)

14: suffer from the same slow convergence issue as the cyclic order in the worst case.  

15: %\peijun{I changed ``Back Substitution" to `back substitution".}

16: % 2a results for CD:

17: In particular, we prove that for unconstrained problems, both GBS-CD and sGS-CD can be $\mathcal{O}(n^2)$ times slower than R-CD.

18: % 2b results for ADMM:

19: \rvs{Despite unconstrained problems,  we also empirically study} linearly constrained problems with quadratic objective: we empirically demonstrate that the convergence speed of GBS-ADMM and sGS-ADMM can be roughly $\mathcal{O}(n^2)$ times slower than randomly permuted ADMM.

20: 

21: 

22: 

23: \end{abstract}

24: