Evaluating the ordering of the SPICE capability levels: an empirical study

The Standard ISO/IEC PDTR 15504 (Software Process Assessment) defines process attributes (PAs) and associated practices must be implemented according to a process capability level. This definition implies that PA practices at lower capability levels must be implemented before moving to higher capability levels. The purpose of this study is to evaluate empirically whether the ordering of set of PAs, as measures of capability, is consistent with the Standard. For this purpose, the study estimates the Coefficient of Reproducibility (CR) statistic that measures the extent to which the observed ratings are identical to the pattern inferred by the Standard. Our analyses based on ratings of 689 process instances show that generally PA order of capability levels is consistent with that inferred by the Standard. However, our results also show that the definition of PA3.2 (Process resource) could be improved. This evaluation is capable of providing a substantiated basis for using the notion of capability, as well as providing information for necessary improvements to the Standard.