Selective multiple kernel learning for classification with ensemble strategy

• We obtain a competitive result with MKL, meanwhile owning sparsity.
• We propose a new kernel evaluation method with quantified result.
• We save the memory to optimize MKL and extend the scale of problem.
• We accelerate MKL optimization by using Lp-norm(p≥2)Lp-norm(p≥2).
• A fast SMKL with L∞-normL∞-norm is proposed, without MKL optimization.

Multiple Kernel Learning (MKL) aims to seek a better result than single kernel learning by combining a compact set of sub-kernels. However, MKL with L1-norm   easily discards the sub-kernels with complementary information and MKL with Lp-norm(p≥2)Lp-norm(p≥2) often gets the redundant solution. To address these problems, a Selective Multiple Kernel Learning (SMKL) method, inspired by Ensemble Learning (EL), is proposed. Comparing MKL with Lp-norm(p≥2)Lp-norm(p≥2), SMKL obtains a sparse solution by a pre-selection procedure. Comparing MKL with L1-norm  , SMKL preserves the sub-kernels with complementary information by guaranteeing the high discrimination and large diversity of pre-selected sub-kernels. For quantifying the discrimination and diversity of sub-kernels, a new kernel evaluation is designed. SMKL reduces the scale of MKL optimization and saves the memory storing of the sub-kernels, which extends the scale of problem that MKL could solve. Specially, a fast SMKL method using L∞-normL∞-norm constraint is focused, which needs no MKL optimization process. It means that the memory is hardly a limitation for MKL with the large scale problem. Experiments state that our method is effective for classification.