Defining and assessing FoI in a large-scale randomized trial: Core components of values affirmation

• We identify the most critical components of an in-classroom values-affirmation intervention.
• We find substantial variation in implementation fidelity across schools and classrooms.
• Tension between fidelity components force teachers to break some implementation protocols.
• Our data support integrating teachers more fully into the process of intervention delivery.
• Significant treatment effects despite some low fidelity suggest not all protocols are necessary.

A growing body of work suggests that values affirmation can serve as a simple, powerful tool for reducing achievement gaps. The dramatic results of these studies have been shared with and discussed by educators, researchers, and policy-makers, spurring excitement about deploying the intervention in schools around the country. Scholars grasp of the mechanism by which the intervention alters student achievement is limited. We develop a framework for assessing fidelity of implementation by identifying the most crucial elements of an ideal classroom-administered values affirmation. We apply this framework to data from a district-wide randomized trial of values affirmation. Our descriptive analysis shows that fidelity varied across schools, teachers, and over time. We believe that out results make a strong case for future implementations to take fidelity into account. Assessment of fidelity of implementation using a critical components framework will ensure better understanding of variation in the impacts of values-affirmation. Our data support integrating teachers more fully into the process of delivery, though we strongly caution that the integration of these written interventions into regular curriculum must be handled carefully. Lastly, even with the threats to the fidelity of delivery and stealth we find significant impacts of the intervention on the intended audience (Borman, Grigg, & Hanselman, 2016). These significant positive impacts despite low fidelity of some criticial components suggests that there may be aspects of the intervention that are less important as they are defined and understood in the original protocol.