64cb7e2b6347f293.tex
1: \begin{abstract}
2: We present \Ac{EL}, a simple yet powerful \ac{DL} algorithm that leverages changing communication topologies to achieve faster model convergence compared to conventional \ac{DL} approaches.
3: At each round of \ac{EL}, each node sends its model updates to a \emph{random sample} of $s$ other nodes (in a system of $n$ nodes).
4: We provide an extensive theoretical analysis of \ac{EL}, demonstrating that its changing topology culminates in superior 
5: convergence properties compared to the state-of-the-art (static and dynamic) topologies.
6: Considering smooth non-convex loss functions, the 
7:  number of  transient iterations for \ac{EL}, 
8: \ie, the rounds required to achieve asymptotic linear speedup, is in $\mathcal{O}(\nicefrac{n^3}{\samplenum^2})$ which outperforms the best-known bound $\mathcal{O}({n^3})$ by a factor of $ s^2 $, indicating the benefit of randomized communication for \ac{DL}.
9: We empirically evaluate \ac{EL}
10: in a 96-node network and compare its performance with state-of-the-art \ac{DL} approaches.
11: Our results illustrate that \ac{EL} converges up to $ 1.7\times $ quicker than baseline \ac{DL} algorithms and attains 2.2\% higher accuracy for the same communication volume.
12: \end{abstract}
13: