On building ensembles of stacked denoising auto-encoding classifiers and their further improvement

- Explores how binarization permits/improves diversification in deep machines.
- Shows the effectiveness of pre-emphasizing samples for deep classification.
- Combines the above with data augmentation to reach record results.
- Opens further research lines in deep learning.

To aggregate diverse learners and to train deep architectures are the two principal avenues towards increasing the expressive capabilities of neural networks. Therefore, their combinations merit attention. In this contribution, we study how to apply some conventional diversity methods -bagging and label switching- to a general deep machine, the stacked denoising auto-encoding classifier, in order to solve a number of appropriately selected image recognition problems. The main conclusion of our work is that binarizing multi-class problems is the key to obtain benefit from those diversity methods.Additionally, we check that adding other kinds of performance improvement procedures, such as pre-emphasizing training samples and elastic distortion mechanisms, further increases the quality of the results. In particular, an appropriate combination of all the above methods leads us to reach a new absolute record in classifying MNIST handwritten digits.These facts reveal that there are clear opportunities for designing more powerful classifiers by means of combining different improvement techniques.