1: \begin{abstract}

2: In recent years, decentralized learning has emerged as a powerful tool not only for large-scale machine learning,

3: but also for preserving privacy.

4: One of the key challenges in decentralized learning is that the data distribution held by each node is statistically heterogeneous.

5: To address this challenge, the primal-dual algorithm called the Edge-Consensus Learning (ECL) was proposed

6: and was experimentally shown to be robust to the heterogeneity of data distributions. 

7: However, the convergence rate of the ECL is provided only when the objective function is convex, 

8: and has not been shown in a standard machine learning setting where the objective function is non-convex.

9: Furthermore, the intuitive reason why the ECL is robust to the heterogeneity of data distributions has not been investigated.

10: In this work, we first investigate the relationship between the ECL and Gossip algorithm

11: and show that the update formulas of the ECL can be regarded as correcting the local stochastic gradient in the Gossip algorithm.

12: Then, we propose the Generalized ECL (G-ECL), which contains the ECL as a special case,

13: and provide the convergence rates of the G-ECL in both (strongly) convex and non-convex settings,

14: which do not depend on the heterogeneity of data distributions.

15: Through synthetic experiments, we demonstrate that the numerical results of both the G-ECL and ECL coincide with the convergence rate of the G-ECL.

16: \end{abstract}

17: