826d882d07f0cb5f.tex
1: \begin{abstract}
2: Decentralized learning has recently been attracting increasing attention for its applications in parallel computation and privacy preservation.
3: Many recent studies stated that the underlying network topology with a faster consensus rate (a.k.a. spectral gap) leads to a better convergence rate and accuracy for decentralized learning.
4: However, a topology with a fast consensus rate, e.g., the exponential graph, generally has a large maximum degree,
5: which incurs significant communication costs.
6: Thus, seeking topologies with both a fast consensus rate and small maximum degree is important.
7: In this study, we propose a novel topology combining both a fast consensus rate and small maximum degree called the \proposed{$\left(k+1\right)$}.
8: Unlike the existing topologies,
9: the \proposed{$(k+1)$} enables all nodes to reach the exact consensus after a finite number of iterations for any number of nodes and maximum degree $k$.
10: Thanks to this favorable property, 
11: the \proposed{$(k+1)$} endows Decentralized SGD (DSGD) with both a faster convergence rate and more communication efficiency than the exponential graph.
12: We conducted experiments with various topologies,
13: demonstrating that the \proposed{$(k+1)$} enables various decentralized learning methods to achieve higher accuracy with better communication efficiency than the existing topologies.
14: Our code is available at \url{https://github.com/yukiTakezawa/BaseGraph}.
15: \end{abstract}
16: