1: \begin{abstract}

2: 	Federated Learning (FL) is an emerging learning paradigm that preserves privacy by ensuring client data locality on edge devices.

3: 	The optimization of FL is challenging in practice due to the diversity and heterogeneity of the learning system.

4: 	Despite recent research efforts on improving the optimization of heterogeneous data, the impact of time-evolving heterogeneous data in real-world scenarios, such as changing client data or intermittent clients joining or leaving during training, has not been well studied.

5: 	In this work, we propose Continual Federated Learning (CFL), a flexible framework, to capture the time-evolving heterogeneity of FL.

6: 	CFL covers complex and realistic scenarios---which are challenging to evaluate in previous FL formulations---by extracting the information of past local datasets and approximating the local objective functions.

7: 	Theoretically, we demonstrate that CFL methods achieve a faster convergence rate than \fedavg in time-evolving scenarios, with the benefit being dependent on approximation quality.

8: 	In a series of experiments, we show that the numerical findings match the convergence analysis, and CFL methods significantly outperform the other SOTA FL baselines.

9: \end{abstract}

10: