A model population analysis method for variable selection based on mutual information

In many fields of chemistry and biology research, the routinely produced analytical data are usually of very high dimension. Thus, variable selection is essential to improve the prediction performance of models and to provide a better understanding of the underlying process that generated the data. Indeed, many kinds of methods have been developed for this purpose, and the variable selection method based on mutual information is one of them, where the relevance between the input variables and the response is maximized and the redundancy of the selected variables is minimized. However, several methods based on mutual information adopt a greedy search path so that the selected variable subset is most likely to be local minimum. To overcome this problem, model population analysis is used to build the search path. Therefore, a novel variable selection method based on information theory combined with model population analysis is proposed in this investigation. Using three real world datasets, the proposed method was tested and further compared with other methods. The results showed that the proposed method achieved competitive performance.

► A novel variable selection method based on mutual information has been proposed.
► Using three real world datasets, the proposed method achieved competitive results.
► The method can be widely applied in many fields, such as genomics and metabolomics.
► The source code of MPA-MMIFS is available at http://code.google.com/p/mpa-mmifs.