Model selection of Gaussian kernel PCA for novelty detection

•A new method is proposed to select Gaussian kernel PCA models.•Kernel PCA reconstruction errors of edge and interior normal samples are used.•Edge and interior normal samples are automatically selected.•The relations of their reconstruction errors are used for model selection.

Model parameters significantly affect model performance. To Gaussian kernel PCA for novelty detection, it is still an open problem to select its kernel parameter and the number of its retained eigenvectors when no novel samples are available. This paper puts forward a model selection criterion for this problem based on the fact that reconstruction errors of edge normal samples should distinguish from those of interior normal samples when model parameters are suitable. Parameters that maximize this criterion are selected as the suitable ones. Extensive experiments are conducted on various data sets, and the results testify the effectiveness of the proposed method.