1: \begin{abstract}

2: We address distributed learning problems, both nonconvex and convex, over undirected networks. In particular, we design a novel algorithm based on the distributed \textit{Alternating Direction Method of Multipliers} (ADMM) to address the challenges of high communication costs, and large datasets.

3: %

4: Our design tackles these challenges i) by enabling the agents to perform multiple local training steps between each round of communications; and ii) by allowing the agents to employ stochastic gradients while carrying out local computations.

5: %

6: We show that the proposed algorithm converges to a neighborhood of a stationary point, for nonconvex problems, and of an optimal point, for convex problems.

7: %

8: We also propose a variant of the algorithm to incorporate variance reduction thus achieving exact convergence. We show that the resulting algorithm indeed converges to a stationary (or optimal) point, and moreover that local training accelerates convergence.

9: %

10: We thoroughly compare the proposed algorithms with the state of the art, both theoretically and through numerical results.

11: \end{abstract}

12: