Optimizing the Gaussian kernel function with the formulated kernel target alignment criterion for two-class pattern classification

Nowadays most of the current kernel learning approaches are showing good results in small datasets and fail to scale to large ones. As such, it is necessary to develop faster kernel optimization algorithms that perform better with larger datasets, especially, for the “Big Data” applications. This paper presents a novel fast method to optimize the Gaussian kernel function for two-class pattern classification tasks, where it is desirable for the kernel machines to use an optimized kernel that adapts well to the input data and the learning tasks. We propose to optimize the Gaussian kernel function by using the formulated kernel target alignment criterion. By adopting the Euler–Maclaurin formula and the local and global extremal properties of the approximate kernel separability criterion, the approximate criterion function can be proved to have a determined global minimum point. Thus, when the approximate criterion function is a sufficient approximation of the criterion function, through using a Newton-based algorithm, the proposed optimization is simply solved without being repeated the searching procedure with different starting points to locate the best local minimum. The proposed method is evaluated on thirteen data sets with three Gaussian-kernel-based learning algorithms. The experimental results show that the criterion function has the determined global minimum point for the all thirteen data sets, the proposed method achieves the best high time efficiency performance and the best overall classification performance.

► Choosing an appropriate kernel is crucial for kernel-based learning algorithms. ► This paper optimizes the Gaussian kernel function rather than the kernel matrix. ► The “kernel target alignment” criterion has been used in many applications. ► The formulated approximate KTA criterion function has a determined global minimum point. ► The proposed method is simple and effective.