On the Kozachenko-Leonenko entropy estimator

- We study in detail the bias and variance of the entropy estimator proposed by Kozachenko and Leonenko (1987) for a large class of densities on Rd.
- We then use the work of Bickel and Breiman (1983) to prove a central limit theorem in dimensions 1 and 2.
- In higher dimensions, we provide a development of the bias in terms of powers of N−2/d.
- This allows us to use a Richardson extrapolation to build, in any dimension, a root-n consistent entropy estimator satisfying a central limit theorem which allows for explicit (asymptotic) confidence intervals.
- To our knowledge, all the previous general root-n consistency results were concerning dimension 1.

We study in detail the bias and variance of the entropy estimator proposed by Kozachenko and Leonenko (1987) for a large class of densities on Rd. We then use the work of Bickel and Breiman (1983) to prove a central limit theorem in dimensions 1 and 2. In higher dimensions, we provide a development of the bias in terms of powers of N−2/d. This allows us to use a Richardson extrapolation to build, in any dimension, a root-n consistent entropy estimator satisfying a central limit theorem which allows for explicit (asymptotic) confidence intervals. To our knowledge, all the previous general root-n consistency results were concerning dimension 1.