Once is not enough: Establishing reliability criteria for feedback and evaluation decisions based on classroom observations

Implementation of effective teacher evaluation procedures is a global challenge in which lowering the chances that teachers receive inaccurate evaluations is a pertinent goal. This study investigates the minimum number of observations required to guarantee that teachers receive feedback with modest reliability (Eρ2 ≥ 0.70) and that any summative decisions about their professional career have high reliability (Eρ2 ≥ 0.90). A sample of 198 classroom observations by 62 colleagues of 69 teachers working at eight schools reveals that reliable feedback requires at least three lesson visits by three different observers and that reliable summative decisions require more than 10 visits. These findings mirror those reported through other observation instruments. This study accordingly offers directions for how schools can implement such procedures most cost-effectively.