Simultaneous Lp-approximation order for neural networks

Simultaneous approximation of a function and its derivatives are required in many science and engineering applications. There have been many studies on the simultaneous approximation capability of feedforward neural networks (FNNs). Most of the studies are, however, only concerned with density or feasibility of performing simultaneous approximation with FNNs, and no quantitative estimation on approximation accuracy of the simultaneous approximation is given. Moreover, all existing density or feasibility results are established in the uniform metric only, and provide no solution to topology specification of the FNNs used. In this paper, by means of the Bernstein-Durrmeyer operator, a class of FNNs is constructed which realize the simultaneous approximation of any smooth multivariate function and all its existing partial derivatives. We present, by making use of multivariate approximation tools, a quantitative upper bound estimation on approximation accuracy of the simultaneous approximation of the FNNs in terms of the modulus of smoothness of the functions to be approximated. The obtained results reveals that the approximation speed of the constructed FNNs depends not only on the number of hidden units used, but also on the smoothness of functions to be approximated.