Learned-loss boosting

This paper considers a problem of jointly estimating a regression function and the distribution of residuals when both are specified non-parametrically. We present a joint penalized optimization criterion that combines log-spline density estimation with spline-based regression methods. We also examine the use of boosting methodology to estimate a regression function over a high dimensional covariate space. We demonstrate that our method has a robustification effect, and show its usefulness in diagnosing problems in data. We illustrate our methods with practical examples when likelihood is an appropriate evaluation criterion.