The use of higher order finite difference schemes is not dangerous

We discuss the issue of choosing a finite difference scheme for numerical differentiation in case the smoothness of the underlying function is unknown. If low order finite difference schemes are used for smooth functions, then the best possible accuracy cannot be obtained. This can be circumvented by using higher order finite difference schemes, but there is concern that this may cause bad error behavior. Here we show, theoretically and by numerical simulation, that this is not the case. However, by doing so, the step-size should be chosen a posteriori.