Construct validity of student perceptions of instructional quality is high, but not perfect: Dimensionality and generalizability of domain-independent assessments

•Multidimensionality of students' instructional quality ratings was confirmed.•Dimensionality was identical in two subjects (English and German lesson).•The postulated model was accepted by a two-level confirmatory factor analysis.•Measurement properties of some scales were comparable across subjects and classes.•The use of ratings for teacher comparisons is more restricted for other scales.

In educational research, student ratings are frequently used to assess aspects of instructional quality. The present study investigates two key aspects of construct validity in student ratings in domain independent assessments: (1) the dimensionality of ratings and (2) their generalizability across classes and two subjects (English and German lessons). A large, representative sample of N = 6909 ninth grade students from 280 classes was used and a structural model postulating five central dimensions of instructional quality (structure, classroom management, understandableness, motivation, student involvement) was tested by means of a two-level confirmatory factor analysis. The five-factor model adequately described the structure of student ratings. In terms of generalizability, the assumption of equal measurement properties of student ratings across classes held for some of the investigated dimensions (structure and classroom management), but not for others (understandableness, motivation, and student involvement). Possible explanations for these differences are finally discussed.