A method to investigate the stability of a sorting map

Many statistical tools are available to check the performance of descriptive analysis panels (alignment, repeatability, and discrimination). There are no equivalent concepts for sorting data. In particular, there is currently no way to clearly assess if the results given by a panel are stable. The main goal of this study is to introduce an indicator of stability of a sorting map, the RVb coefficient, and to assess its relevance. The calculation of the RVb is based on a resampling approach. The stability of sorting maps was assessed using the RVb coefficient on five data sets varying in terms of number of products (8–15) and number of evaluations (10–118). As hypothesized, the RVb coefficient increased with the number of evaluations, decreased with the complexity of the sensory task, and increased with the expertise level of the panelists (which significantly impacted both the repeatability and homogeneity). Therefore, the RVb coefficient was found to be a relevant indicator of stability. A strong recommendation for future sorting studies is to systematically check the stability of the sorting map after data was collected. This can easily be achieved with the RVb coefficient.

► We define an indicator to quantify the stability of sorting maps.
► The RVb stability index is calculated via bootstrapping on panelists.
► Case studies and indications on how to calculate the RVb are presented.
► The number of panelists necessary to reach a stable map varies from study to study.
► The RVb coefficient can be used to quantify stability a posteriori.