A regularized profile likelihood approach to covariance matrix estimation

•Two orthogonally equivariant estimators, the IMMLE and the IMPLE, of the covariance matrix are introduced.•The estimation approach considered is based on modified profile likelihoods.•The proposed estimators are a robust alternative to other estimators when the population covariance structure is unknown and perform well under various loss functions.

Two new orthogonally equivariant estimators of the covariance matrix are proposed. The estimates of the population eigenvalues are isotonized maximum likelihood estimates of the modified profile likelihood obtained from the Wishart distribution, in one case, and of a penalized form of such a likelihood function, in the other, with a penalty that constrains the trace of the sample covariance matrix. Properties of these estimators are studied and numerical risk comparisons with six other well-known estimators are presented to demonstrate the robustness of the proposed estimators for various real and simulated covariance structures.