Performance improvement of deep neural network classifiers by a simple training strategy

Improving the classification performance of Deep Neural Networks (DNN) is of primary interest in many different areas of science and technology involving the use of DNN classifiers. In this study, we present a simple training strategy to improve the classification performance of a DNN. In order to attain our goal, we propose to divide the internal parameter space of the DNN into partitions and optimize these partitions individually. We apply our proposed strategy with the popular L-BFGS optimization algorithm even though it can be applied with any optimization algorithm. We evaluate the performance improvement obtained by using our proposed method by testing it on a number of well-known classification benchmark data sets and by performing statistical analysis procedures on classification results. The DNN classifier trained with the proposed strategy is also compared with the state-of-the-art classifiers to demonstrate its effectiveness. Our classification experiments show that the proposed method significantly enhances the training process of the DNN classifier and yields considerable improvements in the accuracy of the classification results.