1: \begin{abstract}

2: While black-box variational inference is widely used, there is no proof that its stochastic optimization succeeds. We suggest this is due to a theoretical gap in existing stochastic optimization proofs—namely the challenge of gradient estimators with unusual noise bounds,

3: and a composite non-smooth objective.

4: For dense Gaussian variational families, we observe that existing gradient estimators based on reparameterization satisfy a quadratic noise bound and give novel convergence guarantees for proximal and projected stochastic gradient descent using this bound. 

5: This provides the first rigorous guarantee that black-box variational inference converges for realistic inference problems.

6: \end{abstract}

7: