Separation index and partial membership for clustering

We propose a new separation index that measures the magnitude of gaps between any two clusters in a partition, by projecting the data in a pair of clusters into a one-dimensional space in which they have the maximum separation. The resulting projections can also be used to determine partial membership for points near the boundaries between two or more clusters. The matrix of separation indexes is helpful in deciding whether too many or too few clusters are specified in the clustering method.