Visualizing and analyzing convolution neural networks with gradient information

Recently, convolution neural networks (CNNs) have demonstrated state-of-the-art results on image classification. However, our understanding of how they work is still limited. This paper expands the gradient-based visualization method by emphasizing the ability of CNNs in extracting local features, visualizes and analyzes the internal behavior of trained CNNs based on computing the gradient of the class score with respect to every layer of CNNs. Besides, we use a quantitative evaluation named prediction difference to verify the proposed visualization method. The saliency maps constructed from different layers demonstrate what CNNs care about in input images and it turns out that when CNNs classify an image, they identify the location of the object. Finally, we compare AlexNet with VGGnets, all of them have the same focus on the input images and VGGnets are more sensitive to pixels.