1: \begin{abstract}

2: Estimating entropy and mutual information consistently is important for many

3: machine learning applications. The Kozachenko-Leonenko (KL) estimator

4: \citep{kozachenko87statistical} is a widely used nonparametric estimator for

5: the entropy of multivariate continuous random variables, as well as the basis

6: of the mutual information estimator of \citet{Kraskov04estimating}, perhaps the

7: most widely used estimator of mutual information in this setting. Despite the

8: practical importance of these estimators, major theoretical questions regarding

9: their finite-sample behavior remain open. This paper proves finite-sample

10: bounds on the bias and variance of the KL estimator, showing that it achieves

11: the minimax convergence rate for certain classes of smooth functions. In

12: proving these bounds, we analyze finite-sample behavior of $k$-nearest

13: neighbors ($k$-NN) distance statistics (on which the KL estimator is based). We

14: derive concentration inequalities for $k$-NN distances and a general

15: expectation bound for statistics of $k$-NN distances, which may be useful for

16: other analyses of $k$-NN methods.

17: \end{abstract}