Overall and pairwise segregation tests based on nearest neighbor contingency tables

Multivariate interaction between two or more classes (or species) has important consequences in many fields and may cause multivariate clustering patterns such as spatial segregation or association. The spatial segregation occurs when members of a class tend to be found near members of the same class (i.e., near conspecifics) while spatial association occurs when members of a class tend to be found near members of the other class or classes. These patterns can be studied using a nearest neighbor contingency table (NNCT). The null hypothesis is randomness in the nearest neighbor (NN) structure, which may result from–among other patterns–random labeling (RL) or complete spatial randomness (CSR) of points from two or more classes (which is called the CSR independence, henceforth). New versions of overall and cell-specific tests based on NNCTs (i.e., NNCT-tests) are introduced and compared with Dixon’s overall and cell-specific tests and various other spatial clustering methods. Overall segregation tests are used to detect any deviation from the null case, while the cell-specific tests are post hoc pairwise spatial interaction tests that are applied when the overall test yields a significant result. The distributional properties of these tests are analyzed and finite sample performance of the tests are assessed by an extensive Monte Carlo simulation study. Furthermore, it is shown that the new NNCT-tests have better performance in terms of Type I error and power estimates. The methods are also applied on two real life data sets for illustrative purposes.