1: \begin{abstract}

2: %		In this paper, we propose a Gauss-Seidel (G-S) type algorithm for solving a general linear system of equations ${A} x = b$. I

3: Consider the classical problem of solving a general linear system of equations $Ax=b$. It is well known that the (successively over relaxed) Gauss-Seidel scheme and many of its variants may not converge when $A$ is neither diagonally dominant nor symmetric positive definite. Can we have a linearly convergent G-S type algorithm that works for {\it any} $A$? In this paper we answer this question affirmatively by proposing a doubly stochastic G-S algorithm that is provably linearly convergent (in the mean square error sense) for any feasible linear system of equations. The key in the algorithm design is to introduce a {\it nonuniform  double stochastic} scheme for picking the equation and the variable in each update step as well as a stepsize rule. These techniques also generalize to certain iterative alternating projection algorithms for solving the linear feasibility problem $A x\le b$ with an arbitrary $A$, as well as  high-dimensional  minimization problems for training over-parameterized models in machine learning. Our results demonstrate that a carefully designed randomization scheme can make an otherwise divergent G-S algorithm converge.

4: 		

5: %		Our result demonstrates that properly introducing randomization can be vital to ensure the convergence of an (otherwise divergent) G-S type algorithm. To the best of our knowledge, this is the first time that G-S type algorithm is shown to possess global convergence guarantee for linear systems of equalities and inequalities with neither positive definiteness, nor the diagonal dominance of $A$.

6: 	\end{abstract}

7: