1: \begin{abstract}

2: Block-coordinate descent algorithms and alternating minimization methods are fundamental optimization algorithms and an important primitive in large-scale optimization and machine learning. While various block-coordinate-descent-type methods have been studied extensively, only alternating minimization -- which applies to the setting of only two blocks -- is known to have convergence time that scales independently of the least smooth block. A natural question is then: is the setting of two blocks special? 

3: 

4: We show that the answer is ``no'' as long as the least smooth block can be optimized exactly -- an assumption that is also needed in the setting of alternating minimization. We do so by introducing a novel algorithm~\ref{eq:AR-BCD}, whose convergence time scales independently of the least smooth (possibly non-smooth) block. The basic algorithm generalizes both alternating minimization and randomized block coordinate (gradient) descent, and we also provide its accelerated version -- \ref{eq:AAR-BCD}. 

5: 

6: %In this paper, we introduce a novel algorithm~\ref{eq:AR-BCD} for the optimization of smooth convex functions in the first-order-oracle model. Our algorithm generalizes both randomized block-coordinate and alternating minimization methods by allowing exact optimization over a single non-smooth block, and performing randomized block-coordinate gradient descent on the remaining blocks. We also provide an accelerated version of the algorithm, whose oracle complexity matches the best known bounds for block-coordinate descent, but is also able to accommodate exact optimization over a non-smooth block. Finally, we perform an experimental evaluation of our new algorithms and compare their performance with existing methods under different choices of smoothness and size for the different variable blocks.

7: \end{abstract}

8: