Manly transformation in finite mixture modeling

Finite mixture modeling is one of the most rapidly developing areas of statistics due to its modeling flexibility and appealing interpretability. Gaussian mixture models have been popular among researchers for decades proving their usefulness in various applications. However, when Gaussian mixture components do not provide an adequate fit for the data, more general models must be considered. Traditional remedies for deviation from normality include employing a more appropriate distribution as well as transforming data to near-normality. Merging both approaches by introducing a mixture model with components derived from the multivariate Manly transformation is proposed. Such mixture models show good performance in modeling skewness and have excellent interpretability. Forward and backward model selection algorithms are proposed to choose an appropriate multivariate transformation. At each step of these algorithms, a model with the specific combination of skewness parameters is estimated by means of the expectation-maximization algorithm. The developed technique is carefully illustrated on synthetic data and applied to several well-known datasets, with promising results.