A modified Elman neural network with a new learning rate scheme

Elman neural network (ENN) is one of recurrent neural networks (RNNs). Comparing to traditional neural networks, ENN has additional inputs from the hidden layer, which forms a new layer-the context layer. So the standard back-propagation (BP) algorithm used in ENN is called Elman back-propagation algorithm (EBP). ENN can be applied to solve prediction problems of discrete time sequence. However, the EBP algorithm suffers from low convergence speed and poor generalization performance. To solve this problem, a new learning rate scheme is proposed, the convergence of new proposed scheme is proved. Furthermore, the contrast experiment is utilized to demonstrate the effectiveness of the proposed scheme from the aspects of convergence speed and consumption time with some popular schemes such as the original ENN, and PSO-ENN which uses PSO algorithm to search the best structure of ENN. The experience shows that the modified method proposed in this paper works best.