1: \begin{abstract}

2: We study the gradient flow for a relaxed approximation to the Kullback-Leibler (KL) divergence

3: between a moving source and a fixed target distribution.

4: This approximation, termed the

5: KALE (KL Approximate Lower bound Estimator), solves a regularized version of

6: the Fenchel dual problem defining the KL over a restricted class of functions.

7: When using a Reproducing Kernel Hilbert Space (RKHS) to define the function

8: class, we show that the KALE continuously interpolates between the KL and the

9: Maximum Mean Discrepancy (MMD). Like the MMD and other Integral Probability

10: Metrics, the KALE remains well-defined for mutually singular

11: distributions. Nonetheless, the KALE inherits from the limiting KL a greater 

12: sensitivity to mismatch in the support of the distributions, compared with the MMD. These two properties make the

13: KALE gradient flow particularly well suited when the target distribution is supported on a low-dimensional manifold. Under an assumption of sufficient smoothness of the trajectories, we show the global convergence of the KALE flow. We propose a particle implementation of the flow given initial samples from the source and the target distribution, which we use to empirically confirm the KALE's properties.

14: \end{abstract}

15: