Regularized multivariate regression models with skew-t error distributions

•Regularized estimators of the regression parameters and the precision matrix for the multivariate linear regression with skew-t errors.•Developing EM algorithm by taking advantage of the hierarchical representation of the multivariate skew-t distributions.•Using simulation to assess the performance of the iterative algorithms for maximizing the penalized likelihood.•Application to real data analysis.

We consider regularization of the parameters in multivariate linear regression models with the errors having a multivariate skew-t distribution. An iterative penalized likelihood procedure is proposed for constructing sparse estimators of both the regression coefficient and inverse scale matrices simultaneously. The sparsity is introduced through penalizing the negative log-likelihood by adding L1-penalties on the entries of the two matrices. Taking advantage of the hierarchical representation of skew-t distributions, and using the expectation conditional maximization (ECM) algorithm, we reduce the problem to penalized normal likelihood and develop a procedure to minimize the ensuing objective function. Using a simulation study the performance of the method is assessed, and the methodology is illustrated using a real data set with a 24-dimensional response vector.