1: \begin{abstract}

2:   This paper focuses on Bayesian inference in a federated learning context (FL). While several distributed MCMC algorithms have been proposed, few consider the specific limitations of FL such as communication bottlenecks and statistical heterogeneity. Recently, Federated Averaging Langevin Dynamics (\FALD{}) was introduced, which extends the Federated Averaging algorithm to Bayesian inference. We obtain a novel tight non-asymptotic upper bound on the Wasserstein distance to the global posterior for \FALD{}. This bound highlights the effects of statistical heterogeneity, which causes a drift in the local updates that negatively affects convergence. We propose a new algorithm \VRFALDs{} that uses control variates to correct the client drift. We establish non-asymptotic bounds showing that \VRFALDs{} is not affected by statistical heterogeneity.

3:   Finally, we illustrate our results on several FL benchmarks for Bayesian inference.

4: \end{abstract}

5: