Developing a test for exploring student performance in a complex domain: Challenges faced, decisions made, and implications drawn

This paper illustrates the application of existing guidelines to develop a test grounded in theoretical perspectives and empirical findings in the area of problem solving. By documenting this process, the paper outlines the challenges test developers face when seeking to construct a theory/research-driven test, discusses the decisions made at different junctures, and emphasizes that the findings from test administration need to be seen in perspective of those decisions. Moving beyond the guidelines proposed in the literature on test development, the paper also suggests that the results of test administration be linked to instruction and explores how instruction could have informed and might be informed by these findings.

► Developing a theory/research-based test is not a straightforward process.
► The paper documents challenges faced and decisions made during this process.
► Implications of the test administration findings for instruction are considered.