Can Deep Neural Networks Discover Meaningful Pattern Features?

Recent advances in the area of deep neural networks brought a lot of attention to some of the key issues important for their design. In particular for 2D-shapes, their accuracy has been shown to outperform all other classifiers - e.g., in the German Traffic Sign competition run by IJCNN 2011. On the other hand, their training may be quite cumber- some and the structure of the network has to be chosen beforehand. This paper introduces a new sensitivity-based approach capable of picking the right image features from a pre-trained SOM-like feature detector. Experimental results obtained so far for hand-written digit recognition show that pruned network architectures impact a transparent representation of the features actually present in the data while improving network robustness.