FILTWAM - A Framework for Online Affective Computing in Serious Games

This paper introduces a Framework for Improving Learning Through Webcams And Microphones (FILTWAM). It proposes an overarching framework comprising conceptual and technical frameworks for enhancing the online communication skills of lifelong learners. Our approach interprets the emotional state of people using webcams and microphones and combines relevant and timely feedback based upon learner's facial expressions and verbalizations (like sadness, anger, disgust, fear, happiness, surprise, and neutral). The feedback generated from the webcams is expected to enhance learner's awareness of their own behavior. Our research enhances flexibility and scalability in contrast with face- to-face trainings and better helps the interests of lifelong learners who prefer to study at their own pace, place and time. Our small-scale proof of concept study exemplifies the practical application of FILTWAM and provides first evaluation results on that. This study will guide future development of software, training materials, and research. It will validate the use of webcam data for a real-time and adequate interpretation of facial expressions into emotional states. Participants’ behaviour is recorded on videos so that videos will be replayed, rated, annotated and evaluated by expert observers and contrasted with participants’ own opinions in future research.