Evaluating the effectiveness of the Mantel test and Procrustes randomization test for exploratory ecological similarity among paleocommunities

•Paleoecology data are multivariate in nature.•Mantel and PROTEST are powerful, but underutilized multivariate techniques.•We compared these two techniques using real and synthesized data.•Mantel and PROTEST demonstrate similar results with varying types of data.•Researchers should explore both methods when conducting analyses.

The Mantel test and Procrustes Randomization test (PROTEST) are critically valuable to paleoecological research in allowing statistical evaluation of observed trends in multivariate data. Paleoecological data are inherently multivariate, and feature caveats (varied sample size, hierarchical sampling, multiple independent variables) that make them ill-suited for univariate statistics. Established modern ecological cases demonstrate that the Mantel test and PROTEST of multivariate dataset concordance are statistically powerful, versatile, and appropriate for several types of paleontological data. These two multivariate goodness-of-fit tests are used to compare two paleocommunity datasets, to explore how a dataset of environmental variables compares to a paleocommunity dataset, or to test for spatial or temporal autocorrelation. Here, we 1) compare the performance of the Mantel test and PROTEST under different ecological circumstances by conducting a sensitivity analysis on synthesized data. Synthetic data are calibrated to represent a) fossils sampled evenly along an ecological gradient and b) fossils sampled in distinct clusters within different communities. After calibration, we 2) apply the two multivariate statistical tests to published datasets from two paleocommunity studies. The Mantel test and PROTEST consistently demonstrate similar results based on datasets containing samples ranging from clustered to gradient. However, there were some occasions where the Mantel test was more consistent than the PROTEST. It is advisable to explore both methods when conducting analyses, in conjunction with visual, qualitative assessments of ordinations. These tests will enhance the statistical and interpretive potential of contemporary paleoecological research.