The prospects of a quantitative measurement of agility: A validation study on an agile maturity model

•A quantitative measurement of “agility” with connected CI to the items.•A positive result from practitioners on a quantitative agile measurement.•Validation tests of internal consistency and construct validity (negative result).•New groups of items, but questions agile maturity models.•Tradeoff between quick quantitative and time-consuming contextual assessments.

Agile development has now become a well-known approach to collaboration in professional work life. Both researchers and practitioners want validated tools to measure agility. This study sets out to validate an agile maturity measurement model with statistical tests and empirical data. First, a pretest was conducted as a case study including a survey and focus group. Second, the main study was conducted with 45 employees from two SAP customers in the US. We used internal consistency (by a Cronbach’s alpha) as the main measure for reliability and analyzed construct validity by exploratory principal factor analysis (PFA). The results suggest a new categorization of a subset of items existing in the tool and provides empirical support for these new groups of factors. However, we argue that more work is needed to reach the point where a maturity models with quantitative data can be said to validly measure agility, and even then, such a measurement still needs to include some deeper analysis with cultural and contextual items.