1: \begin{abstract}

2: 	In data-parallel optimization of machine learning models, workers collaborate to improve their estimates of the model: more accurate gradients allow them to use larger learning rates and optimize faster.

3: 	In the decentralized setting, in which workers communicate over a sparse graph, current theory fails to capture important aspects of real-world behavior.

4: 	First, the `spectral gap' of the communication graph is not predictive of its empirical performance in (deep) learning.

5: 	Second, current theory does not explain that collaboration enables \emph{larger} learning rates than training alone.

6: 	In fact, it prescribes \emph{smaller} learning rates, which further decrease as graphs become larger, failing to explain convergence dynamics in infinite graphs.

7: 	This paper aims to paint an accurate picture of sparsely-connected distributed optimization.

8: 	We quantify how the graph topology influences convergence in a quadratic toy problem and provide theoretical results for general smooth and (strongly) convex objectives.

9: 	Our theory matches empirical observations in deep learning, and accurately describes the relative merits of different graph topologies. This paper is an extension of the conference paper by~\citet{vogels22beyondSpectral}. Code: \texttt{\href{https://github.com/epfml/topology-in-decentralized-learning}{github.com/epfml/topology-in-decentralized-learning}}.

10: \end{abstract}

11: