On masking and swamping robustness of leading nonparametric outlier identifiers for multivariate data

For any outlier detection procedure, a key concern is robustness with respect to possible misclassification errors, masking (Type I) and swamping (Type II). Although parametric model-based simulation results are informative, one also desires nonparametric masking and robustness measures that are more broadly applicable. To this effect, notions of finite-sample masking and swamping breakdown points formulated abstractly for outlyingness functions in arbitrary data settings (Serfling and Wang, 2014) are introduced in the present paper into the multivariate data setting. Formulas for the measures are derived for three important affine invariant nonparametric multivariate outlyingness functions: Mahalanobis distance, Mahalanobis spatial, and projection. Using the formulas, favorable masking and swamping breakdown points, balanced equally, are seen for the Mahalanobis distance outlyingness using minimum covariance determinant (MCD) location and scatter estimators, and likewise for the projection outlyingness with median and MAD for univariate location and scale. Also, Mahalanobis spatial outlyingness with MCD standardization is competitive when swamping robustness is given higher priority than masking robustness. A small simulation study with bivariate contaminated standard normal and contaminated exponential models yields results consistent with the theoretical formulas. Some practical recommendations are discussed.