Alignment of game design features and state mathematics standards: Do results reflect intentions?

This paper describes the results of a randomized control trial of a standards-based mathematics software on elementary school students' (3rd-5th graders; N = 10,860) mathematics achievement. Spatial Temporal Mathematics (ST Math) engages students by presenting them with a series of game-like activities that are directly tied to the California State Standards for mathematics. We report the effects of the program on students' specific mathematics skills, as well as uncover which elements of the design of the games could be responsible for gains in achievement. We pay particular attention to the alignment of design features of the games, the standards to which developers intended to align their games, and assessments of specific mathematics skills. Results indicate a statistically significant effect of the program on students' basic number sense skills as measured by a standardized measure of mathematics achievement (effect size of 0.14). Subsequent coding of the games for elements that are related to number sense, such as the occurrence of number lines and objects that represent numbers, revealed that these design elements occur throughout the software. We discuss these findings as they relate to the development and design of standards-based mathematics curricula as well as how these features relate to their assessment.