Generalized profiling estimation for global and adaptive penalized spline smoothing

We propose the generalized profiling method to estimate the multiple regression functions in the framework of penalized spline smoothing, where the regression functions and the smoothing parameter are estimated in two nested levels of optimization. The corresponding gradients and Hessian matrices are worked out analytically, using the Implicit Function Theorem if necessary, which leads to fast and stable computation. Our main contribution is developing the modified delta method to estimate the variances of the regression functions, which include the uncertainty of the smoothing parameter estimates. We further develop adaptive penalized spline smoothing to estimate spatially heterogeneous regression functions, where the smoothing parameter is a function that changes along with the curvature of regression functions. The simulations and application show that the generalized profiling method leads to good estimates for the regression functions and their variances.