Application of an organizational evaluation capacity self-assessment instrument to different organizations: Similarities and lessons learned

• This study features a multiple case study design to investigate organizational evaluation capacity in three different organizations.
• An organizational evaluation capacity self-assessment instrument developed by the authors was used to measure EC in the same way across all three organizations.
• The cross-case analysis shows that higher institutionalization of evaluation leads to higher capacity to do and use evaluation.
• However, capacity to do is not required prior to developing capacity to use.

Organizational evaluation capacity (EC) has received significant attention in the evaluation research literature in the past decade. Much of the focus has been on defining organizational evaluation capacity, which can be thought of as the competencies and structures required to conduct high-quality evaluation studies (capacity to do), as well as the organization's ability to integrate evaluation findings into its decision-making processes (capacity to use). This paper seeks to contribute to this growing body of knowledge through a multiple case study of EC across three different organizations (e.g., non-profit, provincial government and federal government, herein named sectors); the novelty of this particular study is that each case study is based on the use of a common measurement tool developed by Bourgeois, Toews, Whynot and Lamarche (2013). The cross-case analysis presented in the paper reveals that evaluation capacity tends to be higher, both in terms of capacity to do and capacity to use, in organizations that have developed systematic mechanisms to institute an evaluation culture within their walls. Interestingly, however, we also found that capacity to use does not first require capacity to do, as evidenced in the non-profit organization under study.