Check-all-that-apply questions: Influence of attribute order on sensory product characterization

Interest in methodologies for sensory product characterization by consumers is increasing, and check-all-that-apply questions (CATA or checklists) have gained some popularity. This research studied bias pertaining to: (i) the order in which sensory attributes are placed within a CATA question, and (ii) the order of a sensory CATA question within an extended product assessment ballot (including product-elicited emotions, purchase intention and overall liking). In three studies including 335 consumers and using fresh fruit as samples, evidence of attribute order bias was established. In each study, two ballot versions were used in a between-subjects design. Primacy effects linked to attribute salience could explain some of the results. For example, differences in frequency of use of an attribute would be higher on the ballot version where it was placed nearer to the top of the list. However, this type of bias was not sufficient to explain all observed effects. It was found that a random ordering of sensory attributes in a CATA question reduced the total frequency of usage of terms compared with when attributes were grouped with similar terms (e.g. flavour/taste terms grouped together and texture terms grouped together). Some evidence was obtained to suggest that listing sensory attributes in the order that approximate the time when they would be perceived during the course of consuming the sample is necessary unless consumers are given explicit instructions to recall all sensory perceptions and evaluate attributes on the list from that point of reference. It was also found that conclusions regarding differences between samples depended on which ballot version was used. Few order effects were uncovered when the sensory CATA question was prior to or subsequent to other product evaluations (product-elicited emotions and purchase intention). There was no significant effect on hedonic scores of the tested products linked to the attribute order within the sensory CATA question. This research can help to inform best practices in the design of CATA questions for sensory product characterization.

► Attribute order bias in sensory CATA questions exists.
► Attribute salience/primacy bias is one contributing factor to order effects.
► Bias affected frequency of use of attributes and conclusions regarding differences between samples.
► Randomly listing sensory attributes in a CATA question reduced frequency of use compared with grouping by sensory modality.