Hopfield neural network: The hyperbolic tangent and the piecewise-linear activation functions

This paper reports two-dimensional parameter-space plots for both, the hyperbolic tangent and the piecewise-linear neuron activation functions of a three-dimensional Hopfield neural network. The plots obtained using both neuron activation functions are compared, and we show that similar features are present on them. The occurrence of self-organized periodic structures embedded in chaotic regions is verified for the two cases.

► We consider a three-dimensional Hopfield neural network.
► The hyperbolic tangent neuron activation function is considered.
► The piecewise-linear neuron activation function is considered.
► Parameter-space plots obtained using both neuron activation functions are compared.
► Self-organized periodic structures are embedded in chaotic regions for both cases.