Modeling by singular value decomposition and the elimination of statistically insignificant coefficients

• Numerical advantages of singular value decomposition over other least squares techniques.
• Elimination of statistically insignificant coefficients.
• Benefit of a statistical rejection procedure.

Singular value decomposition (SVD) has numerical advantages over other least squares modeling techniques because it requires the summation of basis functions only, rather than of their squares and products. It also transforms the original independent variables to an orthogonal system of variables, thus exposing issues of collinearity and singularity. The SVD approach by itself, however, is simply a decomposition of this original matrix of independent variables, and does not refer to observations affected by errors. With no information on observational errors, it does not include a method for rejecting model coefficients that have little statistical significance. Eliminating singular values to reduce model dimensionality in the least squares application of SVD can thus be done on the basis of statistical error tests, a procedure not directly available to many other applications of the SVD method. A statistical backward elimination procedure applied directly to the transformed SVD principal components compares well with a stepwise procedure applied to the original untransformed coordinates, allowing advantage to be taken of the numerical superiority of SVD. On the other hand, it is important to understand that the approaches taken by SVD and ordinary least squares (OLS) in handling singularities are quite different, and in these cases can lead to different solutions. Analyses of several singular and near-singular least squares matrices in the literature, as well as two real-world examples of modeling electric field, demonstrate the similarities and differences between the two least squares approaches, and the benefit of a statistical rejection procedure in both of them.