Accounting for no difference/preference responses or ties in choice experiments

The analysis of choice data in which no difference/preference responses, or ties, occur is considered in this paper. A key issue addressed in the paper is the need for “identicality norms” for difference and preference tests. These norms reflect the researcher’s expectation for the number of ties that would have occurred in the experiment had the products tested been putatively identical. Without these norms, the issue of how to account for ties can never be fully resolved. After this idea is developed, some methods from the statistics literature to account for ties are reviewed and the Thurstonian 2-AC (2-Alternative Choice) model is discussed. Common practices of equal or proportional redistribution of ties are noted to be either conservative or liberal, respectively, when the binomial distribution is used to evaluate results. In particular, the exact probability function for the equal allocation method is given as a particular type of mixing distribution, known as a convolution, of binomial probability functions. Regardless of which statistical method is used to test tied data, however, none of the current methods of analysis can account for segmentation or the effect of heterogeneity in individual assessors. To study the possible effect of heterogeneity, the data could first be tested against an identicality norm. Thus, this research clarifies the assumptions that are made when conducting tests on paired comparison data with ties and provides guidance on the choice of analytic method.

► Guidance in analysis of no difference/preference results is given. ► Need for “identicality norms” identified and explained. ► Common practices of equal and proportional splitting examined. ► Exact distribution for equal splitting provided.