Hopfield neural network: The hyperbolic tangent and the piecewise-linear activation functions

This paper reports two-dimensional parameter-space plots for both, the hyperbolic tangent and the piecewise-linear neuron activation functions of a three-dimensional Hopfield neural network. The plots obtained using both neuron activation functions are compared, and we show that similar features are present on them. The occurrence of self-organized periodic structures embedded in chaotic regions is verified for the two cases.

► We consider a three-dimensional Hopfield neural network. ► The hyperbolic tangent neuron activation function is considered. ► The piecewise-linear neuron activation function is considered. ► Parameter-space plots obtained using both neuron activation functions are compared. ► Self-organized periodic structures are embedded in chaotic regions for both cases.