Investigation of the number of consumers necessary to obtain stable sample and descriptor configurations from check-all-that-apply (CATA) questions

• Stability of sample and term configurations from CATA was studied using bootstrapping.
• The number of consumers necessary to yield stable configurations was determined.
• Stability was affected by degree of difference among samples.
• 60–80 consumers seems reasonable when working with widely different samples.
• A configuration stability index is proposed based on a posteriori bootstrapping.

As the use of check-all-that-apply (CATA) questions for consumers to aid sensory product characterization gains popularity the need for developing guidelines regarding best practice increases. An important issue in the design of studies including CATA questions is the number of consumers that are necessary to obtain stable results. In this context, the aim of the present work was to study the stability of sample and descriptor configurations obtained using CATA questions with consumers, by means of a bootstrapping resampling approach. Data from 13 consumer studies were analyzed, differing in number of consumers, product category, number of samples, degree of difference among samples and number and type of descriptors in the CATA question. For each study, 1000 random subsets of different number of consumers were generated from the original data set. For each consumer virtual panel, sample and descriptor configurations were obtained using Correspondence Analysis and the concordance between them and the reference configurations (obtained with all the consumers) were computed using the regression vector (RV) coefficient. The RV coefficients of sample and descriptor configurations increased with the number of consumers, suggesting an increase in the stability of the configurations. Results showed that the number of consumers necessary to yield fully stable configurations depended on the degree of difference among samples, the complexity of the sensory space and the type of descriptors included in the CATA question. Across the 13 data sets analysed, results suggested that when working with widely different samples, 60–80 consumers can be regarded as a reasonable compromise to get stable sample and descriptor configurations. However, it is recommended to check, a posteriori, the stability of sample and descriptor configurations using a stability index based on bootstrapping.