Short CommunicationRandomization of CATA attributes: Should attribute lists be allocated to assessors or to samples?

- Operational power for different attribute list orders in CATA studies investigated.
- Options: assign attribute list order to assessors or to individual evaluations.
- Constant attribute list order across evaluations of a panelist found to be superior.
- Theoretical considerations support this finding.

For the check-all-that-apply (CATA) question format, it is good practice to vary the attribute list order between evaluations to account for possible (and likely) position bias in the data. If attribute lists are to be randomized, the question is how to allocate these attribute lists orders. Some authors recommend a “to samples” allocation order, randomizing the attribute list order for each sample presented. Other authors recommend a “to assessors” allocation, randomizing the attribute list order for each assessor, such that the list order is stable across samples for each assessor (if replication is used, assessors are given a new attribute list order per replication). In this study, consumers (n = 93) performed CATA evaluations on 6 breads twice. The evaluation was done once using the “to assessors” CATA list order allocation scheme, and once using the “to samples” CATA list order allocation scheme, with the order of allocation schemes balanced by experimental design. Results suggest higher operational power when using the “to assessor” CATA attribute list order. This conclusion is supported by theoretical considerations.