A weighted linear discriminant analysis framework for multi-label feature extraction

Linear discriminant analysis (LDA) is one of the most popular single-label (multi-class) feature extraction techniques. For multi-label case, two slightly different generalized versions have been introduced independently. We argue whether there exists a framework to unify such two multi-label LDA methods and to derive more well-performed multi-label LDA techniques further. In this paper, we build a weighted multi-label LDA framework (wMLDA) to consolidate two existing multi-label LDA-type methods with binary and correlation-based weight forms, and further collect two additional weight forms with entropy and fuzzy principles. To exploit both label and feature information more sufficiently, via maximizing dependence based on Hilbert-Schmidt independence criterion, a novel dependence-based weight form is proposed, which is formulated as a non-convex quadratic programing problem with ℓ1-norm and non-negative constraints and then is solved by random block coordinate descent method with a linear convergence rate. Experiments on ten data sets illustrate that our dependence-based wMLDA works the best, and five wMLDA-type algorithms are superior to canonical correlation analysis and multi-label dimensionality reduction via dependency maximization, according to five multi-label classification performance measures and Wilcoxon statistical test.