A heuristically enhanced gradient approximation (HEGA) algorithm for training neural networks

In this article we study artificial neural network training under the following two conditions: (a) the training algorithm must not rely on direct computation of gradients and (b) the algorithm must be efficient in training on-line. We review various relevant algorithms that are currently available in the literature and we propose a new algorithm that is further improved with respect to the second condition. We test and compare these algorithms by using commonly used benchmark problems in the literature and compare their efficiency against the popular backpropagation algorithm. Also, we introduce a realistic problem incorporating a robotic elbow manipulator and continue testing the algorithms against this problem.