Design and evaluation of instructor-based and peer-oriented attention guidance functionalities in an open source anchored discussion system

• We designed and evaluated two attention guidance functions in an annotation tool.
• Instructor-based function steered students to select challenging concepts from text.
• Instructor-based function led to higher quality interaction on challenging concepts.
• Switching from instructor- to peer-based function did not reduce interaction quality.

Social interactions to supplement learning and asynchronous tools to facilitate exchange of quality ideas have gained much attention in information systems education. While various systems exist, students have difficulty with deep processing of complex instructional materials (e.g., concepts of a theory and pedagogical support mechanisms derived from a theory). This research proposes a theoretical framework that leverages attention guidance in a social constructivist approach to facilitate processing of central domain concepts, principles, and their interrelations. Using an open source anchored discussion system, we designed a set of instructor-based and peer-oriented attention guidance functionalities involving dynamic manipulation of text font size similar to tag clouds. We conducted an experimental study with two small groups of first-year doctoral students in a blended-learning classroom format. Students in the control group had no access to attention guidance functions. Students in the treatment group used instructor-based attention guidance functionality and then switched to peer-oriented attention guidance functionality. The evaluation compared focus, content, and sequential organization of students' online discussion messages with heat maps, content analysis, sequential analysis, and statistical discourse analysis to examine different facets of the phenomenon in a holistic way. The results show that in areas where students struggle to understand challenging concepts, instructor-based attention guidance functionality facilitated elaboration and negotiation of ideas, which is fundamental to higher order thinking. In addition, after switching to peer-oriented attention guidance functionality, students in the treatment group took the lead in pinpointing challenging concepts they did not previously understand. These findings indicate that instructor-based and peer-oriented attention guidance functionalities offer students an indirect way of focusing their attention on deep processing of challenging concepts in an inherently open learning environment. Implications for theory, software design, and future research are discussed.