1: \begin{abstract}

2: In this paper, we consider a distributed stochastic non-convex optimization problem, which is about minimizing a sum of $n$ local cost functions over a network with only zeroth-order information. 

3: A novel single-loop Decentralized Zeroth-Order Variance Reduction algorithm, called DZOVR, is proposed, which combines two-point gradient estimation, momentum-based variance reduction technique, and gradient tracking.

4: Under mild assumptions, we show that the algorithm is able to achieve $\mathcal{O}(dn^{-1}\epsilon^{-3})$ sampling complexity at each node to reach an $\epsilon$-accurate stationary point and also exhibits network-independent and linear speedup properties. 

5: %Under mild assumptions, we show that the algorithm is able to achieve a convergence rate of $\mathcal{O}\left(\left( \frac{d}{nK}\right)^{2/3}\right)$ and also exhibits network-independent linear speedup property. 

6: To the best of our knowledge, this is the first stochastic decentralized zeroth-order algorithm that achieves this sampling complexity.

7: Numerical experiments demonstrate that DZOVR outperforms the other state-of-the-art algorithms and has network-independent and linear speedup properties.

8: \end{abstract}

9: