Clustering boundary detection for high dimensional space based on space inversion and Hopkins statistics

Highlight
• Propose a high dimensional space inversion technique to extract the local space features.
• Propose a Symmetry Statistics to describe the uniformity of high dimensional data space.
• Propose a clustering boundary detection algorithm for high dimensional data space named Spinver.

Physicists research the symmetry of particle space through the contrast of motion law in the real space and inversion space which is created by space inversion techniques. Inspired by this theory, we propose the idea of using local space transformation and dynamic relative position to detect the clustering boundary in high dimensional space. Due to the curse of dimensionality, global space transformation approaches are not only time-consuming, but also fail to keep the original distribution characteristics. So, we inverse the space positions of the k nearest neighbors and project them on the high dimensional space coordinate system. To address the lack of statistics that can describe the uniformity of high dimensional space, we propose the Symmetry Statistics based on the Hopkins Statistics. It is employed to judge the uniformity of k nearest neighbor space of coordinate origin. Moreover, we introduce a filter function to remove some special noises and isolated points. Finally, we use boundary and filter ratios to detect the clustering boundary and propose the corresponding detection algorithm, called Spinver. Experimental results from synthetic and real data sets demonstrate the effectiveness of this algorithm.