917b25693bae6765.tex
1: \begin{abstract}
2: Stochastic gradient descent (SGD) is a promising method for solving large-scale inverse problems, due to its excellent scalability
3: with respect to data size. The current mathematical theory in the lens of regularization theory predicts that SGD
4: with a polynomially decaying stepsize schedule may suffer from an undesirable
5: saturation phenomenon, i.e., the convergence rate does not further improve
6: with the solution regularity index when it is beyond a certain range.
7: In this work, we present a refined convergence rate analysis of SGD, and prove that saturation actually does not occur if the initial
8: stepsize of the schedule is sufficiently small. Several numerical experiments are provided to complement the analysis.\\
9: \textbf{Key words}: stochastic gradient descent; regularizing property; convergence rate; saturation; inverse problems.
10: \end{abstract}