Low-error digital hardware implementation of artificial neuron activation functions and their derivative

In this paper we propose a low-error approximation of the sigmoid function and hyperbolic tangent, which are mainly used to activate the artificial neuron, based on the piecewise linear method. Here, the hyperbolic tangent is alternatively approximated by exploiting its mathematical relationship with the sigmoid function, showing better results. Special attention has been paid to study the minimum number of precision bits to achieve the convergence of a multi-layer perceptron network in finite arithmetic machine. All the approximation results show lower mean relative and absolute error than those reported in the state-of-the-art. Finally, the sigmoid digital implementation is discussed and assessed in terms of work frequency, complexity and error in comparison with the state-of-the-art.

► In this paper we propose a low-error approximation of the sigmoid function. ► An approximation of the hyperbolic tanget is proposed. ► The minimum number of precision bits to achieve the convergence is calculated. ► The obtained results are the lowest in terms of absolute and relative errors.