1: \begin{abstract} 

2: Federated learning (FL) has received a surge of interest in recent years thanks to its benefits in data privacy protection, efficient communication, and parallel data processing.

3: Also, with appropriate algorithmic designs, one could achieve the desirable {\em linear speedup for convergence} effect in FL.

4: However, most existing works on FL are limited to systems with i.i.d. data and centralized parameter servers and results on decentralized FL with heterogeneous datasets remains limited.

5: Moreover, whether or not the linear speedup for convergence is achievable under {\em fully decentralized} FL with data heterogeneity remains an open question.

6: In this paper, we address these challenges by proposing a new algorithm, called NET-FLEET, for fully decentralized FL systems with data heterogeneity.

7: The key idea of our algorithm is to enhance the local update scheme in FL (originally intended for communication efficiency) by incorporating a recursive gradient correction technique to handle heterogeneous datasets.

8: We show that, under appropriate parameter settings, the proposed NET-FLEET algorithm achieves a linear speedup for convergence.

9: We further conduct extensive numerical experiments to evaluate the performance of the proposed NET-FLEET algorithm and verify our theoretical findings.

10: \end{abstract}

11: