A self-organizing deep belief network for nonlinear system modeling

In this paper, a self-organizing deep belief network (SODBN) with growing and pruning algorithms is proposed for nonlinear system modeling. Although deep learning-based DBN has been widely used in recent years, actually more detailed researches about how to dynamically determine its structure are seldom observed in the existing literatures. The SODBN can automatically determine its structure using growing and pruning algorithms instead of artificial experience. Firstly, the structure of SODBN is constructed automatically by changing the number of both hidden layers and the hidden neurons during the training process. The self-organizing strategy is implemented by automatic growing and pruning algorithm (AGP), which is actually equivalent to adding and pruning the connecting weights between neurons. Secondly, the weights are dynamically adjusted during the process of structure self-organization. SODBN is able to adjust the weights in the dynamic process of self-organizing structure, and is helpful to improve the network performances, including running time and accuracy. Finally, the proposed SODBN has been tested on three benchmark problems, including nonlinear system modeling, water quality prediction in practical wastewater treatment system as well as air pollutants concentrations prediction. The corresponding experimental results show that SODBN has better performances than some existing neural networks.