1: \begin{abstract}

2: TO BE REWRITTEN

3: 

4: We revisit three classic algorithms: alternating projection between two linear subspaces, the two-block Kaczmarz method for solving linear equations, and two-block Gauss-Seidel method for least-squares. At each iteration, all three algorithms solve some optimization problem to optimality, thereby inducing the common wisdom that none of them requires any stepsize parameters. On the contrary, we show that empowering these methods with two appropriately chosen stepsizes provably accelerates their vanilla versions. We call this two-block acceleration (TBA). For alternating projection, TBA alleviates the well-known negative result that vanilla alternating projection might be arbitrarily slow. In the case of the two-block Gauss-Seidel method for least-squares, TBA accelerates without momentum, and in some sense beats momentum: We prove TBA is at least twice as fast as the heavy ball method. Overall, the paper provides the first two-step analysis for these problems, as opposed to the prevalent single-step convergence proof in the literature.

5: \end{abstract}

6: