Process mining analysis of conceptual modeling behavior of novices – empirical study using JMermaid modeling and experimental logging environment

• Feedback is usually available to novices when a task is complete.
• This research aims at revealing perspectives of process-oriented feedback.
• We realize this in the context of one learning case – conceptual modelling.
• We used a novel approach by means of process mining of learners’ processes.
• Findings include patterns that establish a basis for process-oriented feedback.

Previous studies on learning challenges in the field of modeling focus on cognitive perspectives, such as model understanding, modeling language knowledge and perceptual properties of graphical notation by novice business analysts as major sources affecting model quality. In the educational context outcome feedback is usually applied to improve learning achievements. However, not many research publications have been written observing the characteristics of a modeling process itself that can be associated with better/worse learning outcomes, nor have any empirically validated results been reported on the observations of modeling activities in the educational context. This paper attempts to cover this gap for conceptual modeling. We analyze modeling behavior (conceptual modeling event data of 20 cases, 10.000 events in total) using experimental logging functionality of the JMermaid modeling tool and process mining techniques. The outcomes of the work include modeling patterns that are indicative for worse/better learning performance. The results contribute to (1) improving teaching guidance for conceptual modeling targeted at process-oriented feedback, (2) providing recommendations on the type of data that can be useful in observing a modeling behavior from the perspective of learning outcomes. In addition, the study provides first insights for learning analytics research in the domain of conceptual modeling.