1: \begin{abstract}

2: {\bf S}tochastic {\bf P}roximal {\bf G}radient (\textbf{SPG}) methods have been widely used for solving optimization problems with a simple (possibly non-smooth) regularizer in machine learning and statistics. However, to the best of our knowledge no non-asymptotic  convergence analysis of SPG exists for non-convex optimization with a {\bf non-smooth and non-convex regularizer}. All existing non-asymptotic  analysis of SPG for solving non-smooth non-convex problems require the non-smooth regularizer to be a convex function, and hence are not applicable to a non-smooth non-convex regularized problem. This work initiates the analysis to bridge this gap and opens the door to non-asymptotic convergence analysis of non-smooth non-convex regularized problems. We analyze several variants of {\bf mini-batch} SPG methods for minimizing a non-convex objective that consists of a smooth non-convex loss and a non-smooth non-convex regularizer. Our contributions are two-fold: (i) we show that they enjoy the same complexities as their counterparts for solving convex regularized non-convex problems in terms of finding an approximate stationary point; (ii) we  develop more practical variants using dynamic mini-batch size instead of a fixed mini-batch size without requiring  the target accuracy level of solution.  The significance of our results  is that they improve  upon the-state-of-art results for solving non-smooth non-convex regularized problems. We also empirically demonstrate the effectiveness of the considered SPG methods in comparison with other peer stochastic methods.

3: \end{abstract}

4: