1: \begin{abstract}

2: While variance reduction methods have shown great success in solving large scale optimization problems, many of them suffer from accumulated errors and, therefore, should periodically require the full gradient computation.

3: In this paper, we present a single-loop algorithm named SLEDGE (Single-Loop mEthoD for Gradient Estimator) for finite-sum nonconvex optimization, which does not require periodic refresh of the gradient estimator but achieves nearly optimal gradient complexity.

4: Unlike existing methods, SLEDGE has the advantage of versatility; (i) second-order optimality, (ii) exponential convergence in the PL region, and (iii) smaller complexity under less heterogeneity of data.

5: 

6: We build an efficient federated learning algorithm by exploiting these favorable properties.

7: We show the first and second-order optimality of the output and also provide analysis under PL conditions.

8: % Moreover, when the local budget is sufficiently large and clients are less (Hessian-)~heterogeneous, the algorithm outperforms existing methods such as FedAvg, SCAFFOLD, and Mime in terms of communication rounds, and requires lower communication complexity than BVR-L-SGD.

9: When the local budget is sufficiently large and clients are less (Hessian-)~heterogeneous, the algorithm requires fewer communication rounds then existing methods such as FedAvg, SCAFFOLD, and Mime.

10: The superiority of our method is verified in numerical experiments.

11: \end{abstract}

12: