Intrinsic dimensionality estimation based on manifold assumption

• Presents a new intrinsic dimension estimation method.
• Geometric relationship between manifold data pints is modeled.
• The proposed method is simple and easy to implementation on large datasets.

Dimensionality reduction is an important tool and has been widely used in many fields of data mining and machine learning. Intrinsic dimension of data sets is a key parameter for dimensionality reduction. In this paper, a new intrinsic dimension estimation method based on geometrical relationship between manifold intrinsic dimension and data neighborhood geodesic distances is presented. The estimator is derived by manifold sampling assumption. On a densely sampled manifold, the number of samples that fall into a ball is equal to the volume times the density of the ball. The radius of the ball is calculated by graph distance which is approximation of geodesic distance on manifold. Then the intrinsic dimension is estimated on each sample. Experiments conducted on synthetic and real world data set show that the performance of our new method is robust and comparable to other works.