Why performance-based contracting failed in Uganda – An “open-box” evaluation of a complex health system intervention

Performance-based contracting (PBC) is a tool that links rewards to attainment of measurable performance targets. Significant problems remain in the methods used to evaluate this tool. The primary focus of evaluations on the effects of PBC (black-box) and less attention to how these effects arise (open-box) generates suboptimal policy learning. A black-box impact evaluation of PBC pilot by the Development Research Group of the World Bank (DRG) and the Ministry of Health (MOH) concluded that PBC was ineffective.This paper reports a theory-based case study intended to clarify how and why PBC failed to achieve its objectives. To explain the observed PBC implementation and responses of participants, this case study employed two related theories i.e. complex adaptive system and expectancy theory respectively.A prospective study trailed the implementation of PBC (2003–2006) while collecting experiences of participants at district and hospital levels.Significant problems were encountered in the implementation of PBC that reflected its inadequate design. As problems were encountered, hasty adaptations resulted in a de facto intervention distinct from the one implied at the design stage. For example, inadequate time was allowed for the selection of service targets by the health centres yet they got ‘locked-in’ to these poor choices. The learning curve and workload among performance auditors weakened the validity of audit results. Above all, financial shortfalls led to delays, short-cuts and uncertainty about the size and payment of bonuses.The lesson for those intending to implement similar interventions is that PBC should not be attempted ‘on the cheap’. It requires a plan to boost local institutional and technical capacities of implementers. It also requires careful consideration of the responses of multiple actors – both insiders and outsiders to the intended change process. Given the costs and complexity of PBC implementation, strengthening conventional approaches that are better attuned to low income contexts (financing resource inputs and systems management) remains a viable policy option towards improving health service delivery.

► An orthodox (black-box) evaluation of performance-based contracting (PBC) showed the experiment to have failed in Uganda.
► Like complex interventions, what got implemented was a dynamic set of components whose effects were masked by orthodox evaluation.
► Weak capacity to design, fund and manage PBC led to ad hoc acts, short-cuts and uncertainty about size and payment of bonuses.
► The feasibility to scale-up and sustain PBC in similar countries is under-cut by the considerable capabilities required to succeed.