The influence of user expertise and prototype fidelity in usability tests

An empirical study examined the impact of user expertise and prototype fidelity on the outcomes of a usability test. User expertise (expert vs. novice) and prototype fidelity (paper prototype, 3D mock-up, and fully operational appliance) were manipulated as independent variables in a 2 × 3 between-subjects design. Employing a floor scrubber as a model product, 48 users carried out several cleaning tasks. Usability problems identified by participants were recorded. Furthermore, performance, system management strategies and perceived usability were measured. The results showed that experts reported more usability problems than novices but these were considered to be less severe than those reported by novices. Reduced fidelity prototypes were generally suitable to predict product usability of the real appliance. The implications for the running of usability tests are specific to the fidelity of the prototype.