1: \begin{abstract}

2: We introduce a novel and efficient algorithm called the stochastic

3: approximate gradient descent (SAGD), as an alternative to the stochastic

4: gradient descent for cases where unbiased stochastic gradients cannot

5: be trivially obtained. Traditional methods for such problems rely

6: on general-purpose sampling techniques such as Markov chain Monte

7: Carlo, which typically requires manual intervention for tuning parameters

8: and does not work efficiently in practice. Instead, SAGD makes use

9: of the Langevin algorithm to construct stochastic gradients that are

10: biased in finite steps but accurate asymptotically, enabling us to

11: theoretically establish the convergence guarantee for SAGD. Inspired

12: by our theoretical analysis, we also provide useful guidelines for

13: its practical implementation. Finally, we show that SAGD performs

14: well experimentally in popular statistical and machine learning problems

15: such as the expectation-maximization algorithm and the variational

16: autoencoders.

17: \end{abstract}

18: