1: \begin{abstract}\vspace{-0.1in}

2: 

3: \noindent\footnote{Published at ICLR 2022. Submission available to public in \url{www.openreview.net} since Sept. 2021.}We study \algo, a distributed optimization framework based on gradient averaging and adaptive AMSGrad algorithm. Gradient compression with error feedback is applied to reduce the communication cost in the gradient transmission process. Our convergence analysis of \algo\ shows that such compressed gradient averaging strategy yields same convergence rate as standard AMSGrad, and also exhibits the linear speedup effect w.r.t. the number of local workers. Compared with recently proposed protocols on distributed adaptive methods, \algo\ is simple and convenient. Numerical experiments are conducted to justify the theoretical findings, and demonstrate that the proposed method can achieve same test accuracy as the full-gradient AMSGrad with substantial communication savings. With its simplicity and efficiency, \algo\ can serve as a useful distributed training framework for adaptive gradient methods.\vspace{-0.1in}

4: \end{abstract}

5: