Introduction of a quality index, based on Generalizability theory, as a measure of reliability for univariate- and multivariate sensory descriptive data

•Generalizability theory, to study the reliability of sensory descriptive data.•Practical example on Asian condiment sauces.•Univariate- and multivariate generalizability coefficients.•Other contributions of the use of Generalizability theory is stated in the article.

In the sensory industry there is a need to quantify the overall usability or reliability of the final results of a sensory study. The sensory field has developed various statistical methods and techniques resulting in graphs or plots aimed at identifying assessors that contribute high error variance. Most of these methods are focused on estimating levels of repeatability, discriminability and agreement. Yet not much attention has gone to estimating the general level of quality of the dataset as a whole. Quality indicators could be helpful to quickly determine which attributes still require panel training, when panel training has been done in a sufficient manner and to determine the usability of the final results.This contribution proposes to use a statistical method, Generalizability theory (G theory), to monitor reliability (or generalizability) (Cronbach, Rajaratnam, & Gleser, 1963; Gleser, Cronbach, & Rajaratnam, 1965; Rajaratnam, Cronbach, & Gleser, 1965) and to study the univariate and multivariate reliability of the dataset. Generalizability theory (G theory) can characterize, disentangle, and estimate all different sources of variation as defined by the investigator. This makes G theory a useful tool for characterizing products, when dealing with multiple different sources of variance in a set of product evaluations. Univariate generalizability coefficients (or univariate quality index) could serve as quality indicators to determine which attributes are sufficiently reliable and in which cases panel training is beneficial for the discriminative ability of the panel or when panel training is redundant. Multivariate generalizability coefficients (or multivariate quality index) could capture reliability of an entire study expressed in one easily interpretable measure and could answer questions related to the overall quality of the experimental design of the study.In this study, a demonstration will be given on how G theory is used to test attribute reliability and to assess panel performance.