Proper and improper use and interpretation of Beta-binomial models in the analysis of replicated difference and preference tests

For the analysis of replicated difference and preference tests, the Beta-binomial (BB) model is frequently used in order to account for possible heterogeneity of the assessors. For two-sided tests, this approach can be quite useful. We argue, however, that a chance-corrected approach should be preferred in most situations employing a one-sided test like, e.g., the triangle test. In randomized one-sided tests, the success probability is bounded below by p0, the success probability from pure guessing. A further development of the BB model accounting for this constraint is supported. Furthermore, we show that heterogeneity found from either a standard BB or a chance-corrected BB model proves that the products are different, i.e., testing for heterogeneity is one way of testing for product differences. The parameters of any BB approach have therefore to be considered in parallel. A simulation study indicates a better performance of the chance-corrected BB model as compared to the standard model.