1: \begin{abstract}

2: % Distributed gradient descent has attracted attention in modern machine learning, especially for handling large datasets. Less focus has been given to the distributed gradient descent

3: % where the partial gradient in each worker is subject to adversarial corruption instead of random noise. In this paper, we explore the challenges of this adversarial setting and propose a distributed gradient descent algorithm, focusing on the robustness against adversarial corruption and noises during model transmission. Furthermore, we derive bounds on the error rates for both non-strongly convex and strongly convex loss functions.

4: Distributed gradient descent algorithms have come to the fore in modern machine learning, especially in parallelizing the handling of large datasets that are distributed across several workers. However, scant attention has been paid to analyzing the behavior of distributed gradient descent algorithms in the presence of adversarial corruptions instead of random noise. In this paper, we formulate a novel problem in which adversarial corruptions are present in a distributed learning system. We show how to use ideas from (lazy) mirror descent to design a corruption-tolerant distributed optimization algorithm.  Extensive convergence analysis for (strongly) convex loss functions is provided for different choices of the stepsize. We carefully optimize the stepsize schedule to accelerate the convergence of the algorithm, while at the same time amortizing the effect of the corruption over time. Experiments based on linear regression, support vector classification, and softmax classification on the MNIST dataset corroborate our theoretical findings. 

5: \end{abstract}

6: