Problem solving learning environments and assessment: A knowledge space theory approach

This paper explores the relation between problem solving learning environments (PSLEs) and assessment concepts. The general framework of evidence-centered assessment design is used to describe PSLEs in terms of assessment concepts, and to identify similarities between the process of assessment design and of PSLE design. We use a recently developed PSLE, the ProNIFA system, to illustrate the concepts of student model, evidence model, and task model, concepts that provide for a close link between problem solving teaching software and assessment concepts. We also introduce ProNIFA because it uses a mathematical method developed in psychometric test theory—Competency-based Knowledge Space Theory (CbKST)—for building a student model based on observations on students' problem solving performance. The experiences made with methods such as CbKST lead us to the conclusion that the time has come to more frequently integrate assessment components into PSLEs, and to use problem solving and simulation environments as part of assessment environments. This will contribute to making assessment more authentic and less obtrusive, and making PSLEs more relevant in formal educational settings.

► Paper identifies in a systematic manner relation between problem solving learning environments (PSLE) and assessment design.
► Analysis existing PSLEs in terms of assessment framework (evidence-centered assessment design).
► Introduces a new PSLE that makes use of cloud computing and a probabilistic student modeling method, knowledge space theory.
► Compares Bayesian networks to knowledge space theory.
►  Takes up Jonassen's challenge that assessment is the weakest link for teaching problem solving.