Feature extraction based on subspace methods for regression problems

In this paper, we propose a couple of new feature extraction methods for regression problems. The first one is closely related to the conventional principle component analysis (PCA) but unlike PCA, it incorporates target information in the optimization process and try to find a set of linear transforms that maximizes the distances between points with large differences in target values. On the other hand, the second one is a regressional version of linear discriminant analysis (LDA) which is very popular for classification problems. We have applied the proposed methods to several regression problems and compared the performance with the conventional feature extraction methods. The experimental results show that the proposed methods, especially the extension of LDA, perform well in many regression problems.