1: \begin{abstract}

2: Federated Learning (FL) is a variant of distributed learning where edge devices collaborate to learn a model without sharing their data with the central server or each other. We refer to the process of training multiple independent models simultaneously in a federated setting using a common pool of clients as multi-model FL. In this work, we propose two variants of the popular FedAvg algorithm for multi-model FL, with provable convergence guarantees. We further show that for the same amount of computation, multi-model FL can have better performance than training each model separately. We supplement our theoretical results with experiments in strongly convex, convex, and non-convex settings.

3: %We finally simulate a real scenario of training nine different image classification models simultaneously and observe that for the same amount of computation, training multiple models simultaneously can lead to better performance than training the models one after the other. 

4: 

5: 

6: %and introduced the Multi-FedAvg Algorithm (MFA) for unbiased client selection in multi-model FL. In this paper we propose two variants of MFA, namely MFA-Rand and MFA-RR, and show that they converge to the global optimum as $\mathcal{O}(1/T)$. We further show that multi-model federated learning theoretically has an advantage over training each model separately. We verify the theoretical results through experiments in strongly convex, convex and non-convex settings. We finally simulate a real scenario of training 9 different image classification models simultaneously and observe a significant gain of Multi-FedAvg over separate runs of single model FedAvg.

7: 

8: \keywords{Federated Learning  \and Distributed Learning \and Optimization}

9: \end{abstract}

10: