Evaluating two ways of calculating priorities in requirements hierarchies – An experiment on hierarchical cumulative voting

When developing large-scale software systems, there is often a large amount of requirements present, and they often reside on several hierarchical levels. In most cases, not all stated requirements can be implemented into the product due to different constraints, and the requirements must hence be prioritized. As requirements on different abstraction levels shall not be compared, prioritization techniques that are able to handle multi-level prioritization are needed. Different such techniques exist, but they seem to result in unfair comparisons when a hierarchy is unbalanced. In this paper, an empirical experiment is presented where an approach that compensate for this challenge is evaluated. The results indicate that some form of compensation is preferred, and that the subjects’ preference is not influenced by the amount of information given.