1: \begin{abstract}

2: 

3: Non-IID data present a tough challenge for federated learning. In this paper, we explore a novel idea of facilitating pairwise collaborations between clients with similar data. We propose FedAMP, a new method employing federated attentive message passing to facilitate similar clients to collaborate more. We establish the convergence of FedAMP for both convex and non-convex models, and propose a heuristic method to further improve the performance of FedAMP when clients adopt deep neural networks as personalized models. Our extensive experiments on benchmark data sets demonstrate the superior performance of the proposed methods.

4: 

5: \nop{

6: Personalized cross-silo federated learning has raised a lot of interest from companies and institutes who want to participate as clients to collaboratively train strong personalized machine learning models without exposing their sensitive private data.

7: The performance of personalized cross-silo federated learning is largely determined by the effectiveness of inter-client collaboration.

8: However, when the data is non-IID across all clients, it is challenging to model the pair-wise collaboration relationships between clients without knowing their private data.

9: In this paper, we propose to tackle this problem by a novel framework named federated attentive message passing (FedAMP). 

10: The key idea is to realize an attentive collaboration mechanism that adaptively discovers the underlying pair-wise collaboration relationships between clients by iteratively encouraging similar clients to collaborate more with each other.

11: We establish the convergence of FedAMP for both convex and non-convex models, and propose a heuristic method to further improve the performance of FedAMP when clients adopt deep neural networks as personalized models. Extensive experiments demonstrate the superior performance of the proposed methods.

12: }

13: \end{abstract}

14: