PeRSIVA: An empirical evaluation method of a student model of an intelligent e-learning environment for computer programming

•The effectiveness and validation of a hybrid student model are evaluated.•A combination of Kirkpatrick's model with the layered evaluation framework are used.•Among results are improvement on students' reactions and learners' performance.•Students' state towards computer programming and e-learning was changed positively.•The student model makes valid and meaningful conclusions and adaptation decisions.

In this paper an evaluation method for assessing the effectiveness, accuracy and validity of a student model was presented. Our method is called PeRSIVA and is a combination of the well-known evaluation method of Kirkpatrick and the layered evaluation framework. These well-known and commonly used evaluation techniques have been selected in order to design an accurate and correct evaluation methodology, since there are no clear guidelines in the literature for the evaluation of the student model of an adaptive tutoring system. Furthermore, PeRSIVA method was used to evaluate the hybrid student model, which combines an overlay model with stereotypes and fuzzy logic techniques, of an e-learning system. Particularly, PeRSIVA assesses the results of student modeling in students' satisfaction, performance, progress, behavior and state, as well as the validity of the conclusions drawn by the student model and the validity of the adaptation decision making. The e-learning system was used by the students of a postgraduate program in the field of informatics in the University of Piraeus and the evaluation results demonstrated learning improvements in students and adaptation success to students' needs.