A novel model of leaky integrator echo state network for time-series prediction

In this paper, an improved leaky integrator echo state network is proposed. The improved model, named double activation functions echo state network (DAF-ESN), introduces double activation functions to replace the original single activation function in the reservoir state update equation. For the purely input-driven applications, the model can flexibly modify the reservoir state according to different input signals. A sufficient condition for DAF-ESN is given to guarantee that DAF-ESN has the echo state property. Since double activation functions are represented as the weighted sum, two new parameters are introduced. The batch gradient descent method is utilized to optimize the parameters. Finally, the proposed method is applied to the time-series prediction. Simulation results show that the DAF-ESN model can greatly improve the accuracy of time-series prediction, while the operating time is not increased.