1: \begin{abstract}

2: Recent advances in Bayesian learning with large-scale data have witnessed emergence 

3: of stochastic gradient MCMC algorithms (SG-MCMC), such as stochastic gradient 

4: Langevin dynamics (SGLD), stochastic gradient Hamiltonian MCMC (SGHMC), and 

5: the stochastic gradient thermostat. While finite-time convergence properties of 

6: the SGLD with a 1st-order Euler integrator have recently been studied, corresponding theory for 

7: general SG-MCMCs has not been explored. In this paper we consider general SG-MCMCs 

8: with high-order integrators, and develop theory to analyze finite-time convergence properties 

9: and their asymptotic invariant measures. Our theoretical results show faster convergence 

10: rates and more accurate invariant measures for SG-MCMCs with higher-order integrators. 

11: For example, with the proposed efficient 2nd-order symmetric splitting integrator, the 

12: {\em mean square error} (MSE) of the posterior average for the SGHMC achieves an optimal 

13: convergence rate of $L^{-4/5}$ at $L$ iterations, compared to $L^{-2/3}$ for the SGHMC and 

14: SGLD with 1st-order Euler integrators. Furthermore, convergence results of decreasing-step-size 

15: SG-MCMCs are also developed, with the same convergence rates as their fixed-step-size 

16: counterparts for a specific decreasing sequence. Experiments on both synthetic and real 

17: datasets verify our theory, and show advantages of the proposed method in two large-scale 

18: real applications.

19: \end{abstract}

20: