Generic chaining and the ℓ1-penalty

We address the choice of the tuning parameter λλ in ℓ1-penalizedℓ1-penalized M-estimation. Our main concern is models which are highly non-linear, such as the Gaussian mixture model. The number of parameters p is moreover large, possibly larger than the number of observations n. The generic chaining technique of Talagrand (2005) is tailored for this problem. It leads to the choice λ≈logp/n, as in the standard Lasso procedure (which concerns the linear model and least squares loss).

► We generalize the Lasso methodology to high-dimensional non-linear models.
► We show a new application of generic chaining and the Fernique–Slepian theorem.
► We present a sharp oracle inequality for M-estimators.