Adaptive multi-objective swarm fusion for imbalanced data classification

Learning a classifier from an imbalanced dataset is an important problem in data mining and machine learning. Since there is more information from the majority classes than the minorities in an imbalanced dataset, the classifier would become over-fitted to the former and under-fitted to the latter classes. Previous attempts to address the problem have been focusing on increasing the learning sensitivity to the minorities and/or rebalancing sample sizes among classes before learning. However, how to efficiently identify their optimal mix in rebalancing is still an unresolved problem. Due to non-linear relationships between attributes and class labels, merely to rebalance sample sizes rarely comes up with optimal results. Moreover, brute-force search for the perfect combination is known to be NP-hard and hence a smarter heuristic is required. In this paper, we propose a notion of swarm fusion to address the problem - using stochastic swarm heuristics to cooperatively optimize the mixtures. Comparing with conventional rebalancing methods, e.g., linear search, our novel fusion approach is able to find a close to optimal mix with improved accuracy and reliability. Most importantly, it has found to be with higher computational speed than other coupled swarm optimization techniques and iteration methods. In our experiments, we first compared our proposed solution with traditional methods on thirty publicly available imbalanced datasets. Using neural network as base learner, our proposed method is found to outperform other traditional methods by up to 69% in terms of the credibility of the learned classifiers. Secondly, we wrapped our proposed swarm fusion method with decision tree. Notably, it defeated six state-of-the-art methods on ten imbalanced datasets in all evolution metrics that we considered.