Evolutionary pre-training for CRJ-type reservoir of echo state networks

Echo state network (ESN) had emerged as an alternative to recurrent neural network (RNN). Currently in practice, the reservoir of ESN is generated randomly, and manual tuning of its parameters is required for best performance. In this paper we present an adaptation algorithm for the reservoir parameters or “pre-training” of a recently proposed reservoir type called deterministically constructed cycle reservoir with regular jumps (CRJ). Pre-training is achieved by maximizing the average mutual information between the states of the reservoir׳s neurons and the desired response. Our method does not train the readout during the pre-training process, thus it has much lower computational cost than supervised pre-training. Reservoirs pre-trained with our method outperformed or have comparable performance to those generated using parameters reported in [1], obtained by supervised brute-force search of the reservoir parameter space.