Impacts of electronic process guides by types of user: An experimental study

• Our research investigates the perceptions on several usability metrics on Electronic Process Guides (EPGs) through an experimental repeated-measure design with an international sample collected from 4 countries.
• Our research advances on two previous similar studies (of the four only found in the top literature) regarding data collected from a more professional and specialized sample vs undergraduate students or full-time graduate students used in these other two papers.
• Our research helps to understand the non-expected results reported in previous experimental studies on EPGs through the differentiation of types of EPGs and types of Users (practitioners vs academicians; and novices vs experts).

The design and utilization of Electronic Process Guides (EPGs) have been studied in Software Engineering (SwE) since the 1990s. However, the empirical findings from surveys, case studies, and experiments on the beneficial effects of their utilization are still lacking. Thus, we suggest that further research on the utilization of EPGs is required. In this study, we are interested in gaining insights on the effects of using EPGs on objective metrics (learning score, time effort) and subjective metrics (perceived usefulness, ease of use, and value), by comparing three EPG designs (a simple PDF-based EPG, a normal HTML-based EPG, and a sophisticated Java-based EPG) with different blocks of experimental subjects (practitioners, academicians, novices, and experts). To this end, we have conducted a controlled experiment with a sample of international participants in the domain of IT Service Management. We found that the utilization of EPGs improves the objective metrics while no improvements were perceived on the subjective ones, and that the sophisticated EPG design is more appropriate for the academic and expert types of users than for the practitioner and novice types. Thus, our main recommendation for the design and utilization of EPGs is to consider the type of end-user.