On the evaluation of fit measures for quasi-orders

There are several measures available, which calculate the fit between a quasi-order on a set of dichotomous items and a set of observed response patterns. We investigate how we can evaluate such measures of fit for quasi-orders concerning their adequacy. The most prominent use cases for such fit measures are explorative data analysis and the empirical comparison of existing quasi-orders, for example obtained by querying experts. We formulate several requirements, which a fit measure for quasi-orders must fulfil to be useful in these cases. Two fit measures used in explorative data analysis are the correlational agreement coefficient (van Leeuwe, 1974) and the diff-coefficient (Schrepp, 1999). We investigate in several simulation studies if these fit measures fulfil the described requirements.