Using planning poker for combining expert estimates in software projects

When producing estimates in software projects, expert opinions are frequently combined. However, it is poorly understood whether, when, and how to combine expert estimates. In order to study the effects of a combination technique called planning poker, the technique was introduced in a software project for half of the tasks. The tasks estimated with planning poker provided: (1) group consensus estimates that were less optimistic than the statistical combination (mean) of individual estimates for the same tasks, and (2) group consensus estimates that were more accurate than the statistical combination of individual estimates for the same tasks. For tasks in the same project, individual experts who estimated a set of control tasks achieved estimation accuracy similar to that achieved by estimators who estimated tasks using planning poker. Moreover, for both planning poker and the control group, measures of the median estimation bias indicated that both groups had unbiased estimates, because the typical estimated task was perfectly on target. A code analysis revealed that for tasks estimated with planning poker, more effort was expended due to the complexity of the changes to be made, possibly caused by the information provided in group discussions.