1: \begin{abstract}

2: The recently proposed \textit{continuous stochastic gradient descent} (CSG) scheme represents an efficient method for the solution of a large class of stochastic optimization problems. During the course of the iterations, CSG uses a tailored convex combination of previously drawn gradient and objective function samples to approximate the full gradient and objective function value at the current position. It was shown that the error in these approximations tends to zero during the optimization process and thus, CSG more and more behaves like a full gradient descent algorithm. In this contribution, we show that convergence results for constant step sizes known from full gradient descent schemes carry over to CSG. Furthermore, we augment CSG by an Armijo-type backtracking line search based on the gradient and objective function approximations and provide convergence results for this variant. The resulting novel methods are applicable for a large class of optimization problems. Their performance is demonstrated in comparison to other stochastic gradient schemes known from literature.

3: \keywords{Stochastic gradient scheme \and Convergence analysis \and Step size rule \and Backtracking line search \and Constant step size}

4: % \PACS{PACS code1 \and PACS code2 \and more}

5: \subclass{65K05 \and 90C06 \and 90C15 \and 90C30}

6: \end{abstract}

7: