1: \begin{abstract}

2: % Federated learning enables several distributed nodes to train a global model collaboratively. However, the typical federated learning paradigm faces the problem of parameter optimization. Each time the central server of federated learning wants to debug the hyperparameter, it has to restart the training process of federated learning, which brings huge communication and computation costs to all the participants. \hwc{this idea flow is good, but it reads too negative, let's phrase it in a way that "FL is good $\to$ here is something to make it better", instead of the current one "this is FL $\to$ it has issues"}

3: % To solve this problem, we introduce a novel distributed data distillation architecture named DistDD (Distributed Data Distillation through gradient matching) in this paper. DistDD is a new approach that combines gradient matching methods with a distributed learning framework to extract distilled knowledge from all clients. 

4: % This approach can extract a global distilled dataset from all the participated nodes. By acquiring the dataset, the central server can iteratively optimize the trained model without restarting the whole federated learning process.

5: % We further provide a convergence proof of the DistDD algorithm, which provides a solid mathematical foundation for stability and reliability in practical applications. Finally, we demonstrate the convergence of DistDD by conducting detailed experiments in the non-i.i.d. and mislabeling cases. 

6: % These experimental results verify the effectiveness and robustness of DistDD in dealing with complex real-world situations. 

7: % Overall, by aggregating decentralized data insights without actual data transfer, DistDD addresses the challenges of the debugging issues of federated learning. This methodology not only enhances the robustness and diversity of the dataset but also paves the way for scalable and efficient machine-learning models in distributed environments.

8: % \end{abstract}

9: