1: \begin{abstract}

2:         In this paper, we propose a novel reinforcement-learning algorithm consisting in a stochastic variance-reduced version of policy gradient for solving Markov Decision Processes (MDPs).

3:         Stochastic variance-reduced gradient (SVRG) methods have proven to be very successful in supervised learning.

4:         However, their adaptation to policy gradient is not straightforward and needs to account for I) a non-concave objective function; II) approximations in the full gradient computation; and III) a non-stationary sampling process.

5:         The result is SVRPG, a stochastic variance-reduced policy gradient algorithm that leverages on importance weights to preserve the unbiasedness of the gradient estimate.

6:         Under standard assumptions on the MDP, we provide convergence guarantees for SVRPG with a convergence rate that is linear under increasing batch sizes.

7:         Finally, we suggest practical variants of SVRPG, and we empirically evaluate them on continuous MDPs.

8: \end{abstract}

9: