1: \begin{abstract}

2: The Black Box Variational Inference (\citet{ranganath2014black})

3: algorithm provides a universal method for Variational Inference, but

4: taking advantage of special properties of the approximation family

5: or of the target can improve the convergence speed significantly.

6: For example, if the approximation family is a transformation family,

7: such as a Gaussian, then switching to the reparameterization gradient

8: (\citet{kingma2013auto}) often yields a major reduction in gradient

9: variance. Ultimately, reducing the variance can reduce the computational

10: cost and yield better approximations.

11: 

12: We present a new method to extend the reparameterization trick to

13: more general exponential families including the Wishart, Gamma, and

14: Student distributions. Variational Inference with Numerical Derivatives

15: (VIND) approximates the gradient with numerical derivatives and reduces

16: its variance using a tight coupling of the approximation family. The

17: resulting algorithm is simple to implement and can profit from widely

18: known couplings. Our experiments confirm that VIND effectively decreases

19: the gradient variance and therefore improves the posterior approximation

20: in relevant cases. It thus provides an efficient yet simple Variational

21: Inference method for computing non-Gaussian approximations.

22: \end{abstract}

23: