Entropy and orthogonality based deep discriminative feature learning for object recognition

Inspired by the class-selectivity of the neurons in the inferior temporal (IT) area of the human visual cortex, we propose a novel discriminative feature learning method to improve the object recognition performance of convolutional neural network (CNN) without increasing the network complexity. Specifically, we apply the proposed entropy-orthogonality loss (EOL) to the penultimate layer of the CNN models in the training phase. The EOL explicitly enables the feature vectors learned by a CNN model have the following properties: (1) each dimension of the feature vectors only responds strongly to as few classes as possible, and (2) the feature vectors from different classes are as orthogonal as possible. When combined with the softmax loss, the EOL not only can enlarge the differences in the between-class feature vectors, but also can reduce the variations in the within-class feature vectors. Therefore, the discriminative ability of the learned feature vectors is highly improved. The EOL is general and independent of the CNN structure. Comprehensive experimental comparisons with both the image classification and face verification task on several benchmark datasets demonstrate that utilizing the proposed EOL during training can remarkably improve performance of CNN models compared to the corresponding baseline models trained without utilizing the EOL.