1: \begin{abstract}

2: % Federated learning (FL) allows many clients to collaboratively train a model without gathering data to a central node. In practice, data distribution usually varies across clients. Prior works~\cite{collins2021exploiting,arivazhagan2019federated} propose to learn a globally shared encoder and multiple predictors per client locally to overcome such heterogeneity. However, such a client-wise personalized strategy ignores the intrinsic connection between the heterogeneous data distributions across clients. In this work, we consider a more realistic setting for personalized FL, where we assume each client's data distribution is a mixture of several predefined domains. Under this setting, we theoretically show that the domain-aware personalized FL algorithm is superior to the domain-unaware algorithm which treats each client separately. We provide a novel federated learning framework, \our, that is able to learn a shared representation across domains and domain-wise personalized heads with the consideration of fairness across domains. We theoretically analyze the convergence of our algorithm and empirically demonstrate its superiority compared to multiple baselines on both synthetic and real-world datasets.

3: % \end{abstract}

4: