Fuzzy Linear Discriminant Analysis-guided maximum entropy fuzzy clustering algorithm

Linear Discriminant Analysis (LDA) is a classical statistical approach for supervised feature extraction and dimensionality reduction, hard c-means (HCM) is a classical unsupervised learning algorithm for clustering. Based on the analysis of the relationship between LDA and HCM, Linear Discriminant Analysis-guided adaptive subspace hard c-means clustering algorithm (LDA–HCM) had been proposed. LDA–HCM combines LDA and HCM into a coherent framework and can adaptively reduce the dimension of data while performing data clustering simultaneously. Seeing that LDA–HCM is still a hard clustering algorithm, we consider the fuzzy extension version of LDA–HCM in this paper. To this end, firstly, we propose a new optimization criterion of Fuzzy Linear Discriminant Analysis (FLDA) by extending the value of membership function in classical LDA from binary 0 or 1 into closed interval [0, 1]. In the meantime, we present an efficient algorithm for the proposed FLDA. Secondly, we show the close relationship between FLDA and Maximum Entropy Fuzzy Clustering Algorithm (MEFCA): they both are maximizing fuzzy between-class scatter and minimizing within-class scatter simultaneously. Finally, based on the above analysis, combining FLDA and MEFCA into a joint framework, we propose fuzzy Linear Discriminant Analysis-guided maximum entropy fuzzy clustering algorithm (FLDA–MEFCA). LDA–MEFCA is a natural and effective fuzzy extension of LDA–HCM. Due to the introduction of soft decision strategy, FLDA–MEFCA can yield fuzzy partition of data set and is more flexible than LDA–HCM. We also give the convergence proof of FLDA–MEFCA. Extensive experiments on a collection of benchmark data sets are presented to show the effectiveness of the proposed algorithm.

► We propose a new FLDA and present an efficient algorithm for it. ► We show the objectives of FLDA and MEFCA are the same. ► We combine FLDA and MEFCA into a joint clustering framework. ► We present the convergence proof of FLDA-guided MEFCA.