Generation of compound features based on feature interaction for classification

Dimensionality Reduction (DR) is an important preprocessing step in supervised learning. It involves reducing/mapping of high-dimensional data into a low-dimensional space by preserving important information. The classical DR paradigm can be divided into Feature Selection (FS) and Feature Extraction (FE) approaches. So far, these approaches have been studied extensively but independently and the reduced set contains either original or transformed features. And, it is well known that FS and FE approaches based on information theoretic measures are considered to be the most effective approaches as these measures are able to compare the nonlinear relationships between random variables. Herein, we present a novel scheme to generate reduced compound (both original and transformed) feature set based on such measures in supervised learning. This method considers information theoretic measure Mutual Information (MI) and MI based interactions between features and it is able to produce maximum informative and less redundant compound features. The performance of the proposed algorithm is compared with state-of-the-art DR methods using multiple classifiers on UCI machine learning repository and face and object recognition and bio-microarray data sets.