Data smoothing and numerical differentiation by a regularization method

While data smoothing by regularization is not new, the method has been little used by scientists and engineers to analyze noisy data. In this tutorial survey, the general concepts of the method and mathematical development necessary for implementation for a variety of data types are presented. The method can easily accommodate unequally spaced and even non-monotonic scattered data. Methods for scaling the regularization parameter and determining its optimal value are also presented. The method is shown to be especially useful for determining numerical derivatives of the data trend, where the usual finite-difference approach amplifies the noise. Additionally, the method is shown to be helpful for interpolation and extrapolation. Two examples data sets were used to demonstrate the use of smoothing by regularization: a model data set constructed by adding random errors to a sine curve and global mean temperature data from the NASA Goddard Institute for Space Studies.