An alternate method between generative objective and discriminative objective in training classification Restricted Boltzmann Machine

As a derivative of Restricted Boltzmann Machine (RBM), classification RBM (ClassRBM) has been an effective classifier. However, there are still many disadvantages in training ClassRBM. For example, the prediction accuracy with the generative objective function (GenF) is not high, and the training process with the discriminative objective function (DisF) and the hybrid RBM (HDRBM) are time-consuming. In this paper, we propose an alternate method between Generative Objective and Discriminative Objective (ANGD) to train ClassRBM after examining the training process of GenF and DisF. At each iteration step of ANGD, the parameters of ClassRBM are firstly updated by maximizing GenF when the training accuracy can be improved, then modified by maximizing DisF. This process is repeated until some stop criterion is met. ANGD achieves a good prediction accuracy with a relatively less training cost because it utilizes the complementation of GenF and DisF. The comparative experiments on five datasets show that ANGD beats GenF, DisF and HDRBM. As a whole, the accuracy of ANGD is the best and the stability is acceptable, and the training cost of ANGD is also the best on the datasets with a large size. The training efficiency of ANGD is the best among the four methods.