Improving deep ensemble vehicle classification by using selected adversarial samples

Most image classification algorithms aim to maximize the percentage of class labels that are predicted correctly. These algorithms often missclassify images from minority categories as into the dominant categories. To overcome the issue of unbalanced data for classifying vehicles from traffic surveillance images, we propose a semi supervised pipeline focused on integrating deep neural networks with data augmentation based on generative adversarial nets (GANs). The proposed approach consists of three main stages. In the first stage, we trained several GANs on the original dataset to generate adversarial samples for the rare classes. In the second stage, an ensemble of CNN models with different architectures are trained on the original imbalanced data set, and then a sample selection step is performed to filter out the low-quality adversarial samples. In the final stage, the aforementioned ensemble model is refined on the augmented dataset by adding the selected adversarial samples. Experiments on the highly imbalanced large benchmark “MIOvision Traffic Camera Dataset (MIO-TCD)” classification challenge dataset demonstrate that the proposed framework is able to increase the mean performance of some categories to some extent, while maintaining a high overall accuracy, compared with the baseline.