1: \begin{abstract}

2: Federated learning, a decentralized approach to machine learning, faces significant challenges such as extensive communication overheads, slow convergence, and unstable improvements.

3: These challenges primarily stem from the gradient variance due to heterogeneous client data distributions.

4: To address this, we introduce a novel Networked Control Variates (FedNCV) framework for Federated Learning.

5: We adopt the REINFORCE Leave-One-Out (RLOO) as a fundamental control variate unit in the FedNCV framework, implemented at both client and server levels.

6: At the client level, the RLOO control variate is employed to optimize local gradient updates, mitigating the variance introduced by data samples.

7: Once relayed to the server, the RLOO-based estimator further provides an unbiased and low-variance aggregated gradient, leading to robust global updates.

8: This dual-side application is formalized as a linear combination of composite control variates. We provide a mathematical expression capturing this integration of double control variates within FedNCV and present three theoretical results with corresponding proofs.

9: This unique dual structure equips FedNCV to address data heterogeneity and scalability issues, thus potentially paving the way for large-scale applications.

10: Moreover, we tested FedNCV on six diverse datasets under a Dirichlet distribution with \( \alpha = 0.1 \), and benchmarked its performance against six SOTA methods, demonstrating its superiority.

11: \end{abstract}

12: