1: \begin{abstract}
2: In this paper, we consider solving the distributed optimization problem over a multi-agent network under the communication restricted setting. We study a compressed decentralized stochastic gradient method, termed ``compressed exact diffusion with adaptive stepsizes (CEDAS)", and show the method asymptotically achieves comparable convergence rate as centralized SGD for both smooth strongly convex objective functions and smooth nonconvex objective functions under unbiased compression operators.
3: % This guarantees that CEDAS can indeed save communication compared to its centralized counterpart.
4: In particular, to our knowledge, CEDAS enjoys so far the shortest transient time (with respect to the graph specifics) for achieving the convergence rate of centralized SGD, which behaves as $\order{nC^3/(1-\lambda_2)^{2}}$ under smooth strongly convex objective functions, and $\order{n^3C^6/(1-\lambda_2)^4}$ under smooth nonconvex objective functions, where $(1-\lambda_2)$ denotes the spectral gap of the mixing matrix, and $C>0$ is the compression-related parameter. Numerical experiments further demonstrate the effectiveness of the proposed algorithm.
5: % For smooth nonconvex objective functions, the derived transient time behaves as $\order{n^3C^6/(1-\lambda_2)^4}$. Our results can also apply to biased compression operators with the help of so-called introduced compressor.
6: \end{abstract}
7: