New indices for cluster validity assessment

Cluster validation is a technique for finding a set of clusters that best fits natural partitions (of given datasets) without the benefit of any a priori class information. A cluster validity index is used to validate the outcome. This paper presents an analysis of design principles implicitly used in defining cluster validity indices and reviews a variety of existing cluster validity indices in the light of these principles. This includes an analysis of their design and performance. Armed with a knowledge of the limitations of existing indices, we proceed to remedy the situation by proposing six new indices. The new indices are evaluated through a series of experiments.