Self-organization and lateral interaction in echo state network reservoirs

Echo state networks (ESNs) are recurrent structures that give rise to an interesting trade-off between achievable performance and tractability. This is a consequence of the fact that the key element of these networks – the recurrent intermediate layer known as dynamical reservoir – is not, as a rule, subject to supervised training, which is restricted to the linear output layer, also termed as readout. This trade-off, aside from being of theoretical significance, establishes ESNs as most attractive tools for both online and offline information processing. There are two key aspects to be taken into account in the ESN design: (i) the unsupervised definition of the synaptic weights of the reservoir and (ii) the definition of the structure and of the training strategy associated with the readout. This work is concerned with the first of these aspects: it proposes novel strategies for ESN reservoir design based on the theoretical framework built by Kohonen׳s classical works on self-organization – which includes the notions of short-range positive feedback and lateral inhibition – and also on the related and more recent notion of neural gas. It is shown, with the aid of a representative set of simulation results, that the proposed methodologies are capable of leading to significant performance improvements in the context of relevant information processing tasks – channel equalization and chaotic time series prediction – particularly when the input data suits well a cluster-based profile.