1: \begin{abstract}            % Abstract of not more than 250 words.

2: 

3: This paper investigates the stochastic distributed nonconvex optimization problem of minimizing a global cost function formed by the summation of $n$ local cost functions. We solve such a problem by involving zeroth-order (ZO) information exchange.

4: In this paper, we propose a \underline{ZO} \underline{d}istributed pr\underline{i}mal--du\underline{a}l \underline{c}oordinate method ~(ZODIAC) to solve the stochastic optimization problem. 

5: Agents approximate their own local stochastic ZO oracle along with coordinates with an adaptive smoothing parameter.

6: %

7: %

8: We show that the proposed algorithm achieves the convergence rate of $\mathcal{O}(\sqrt{p}/\sqrt{T})$ for general nonconvex cost functions.

9: %

10: %

11: We demonstrate the efficiency of proposed algorithms through a numerical example in comparison with the existing state-of-the-art centralized and distributed ZO algorithms.

12: % We show that the proposed algorithms achieve the linear speedup convergence rate $\mathcal{O}(\sqrt{p/(nT)})$ for smooth cost functions and an $\mathcal{O}(p/(nT))$ convergence rate when the global cost function satisfies the Polyak--{\L}ojasiewicz (P--{\L}) condition in addition, where $p$ and $T$ are the dimension of the decision variable and the total number of iterations, respectively. To the best of our knowledge, this is the first linear speedup result for distributed ZO algorithms, which enables us to scale out the computing capability by adding more agents. We also show that the proposed algorithms converge linearly when considering deterministic centralized optimization problems under the P--{\L} condition.

13: \end{abstract}

14: