Too good to be bad: Favorable product expectations boost subjective usability ratings

In an experiment conducted to study the effects of product expectations on subjective usability ratings, participants (N = 36) read a positive or a negative product review for a novel mobile device before a usability test, while the control group read nothing. In the test, half of the users performed easy tasks, and the other half hard ones, with the device. A standard usability test procedure was utilized in which objective task performance measurements as well as subjective post-task and post-experiment usability questionnaires were deployed. The study revealed a surprisingly strong effect of positive expectations on subjective post-experiment ratings: the participants who had read the positive review gave the device significantly better post-experiment ratings than did the negative-prime and no-prime groups. This boosting effect of the positive prime held even in the hard task condition where the users failed in most of the tasks. This finding highlights the importance of understanding: (1) what kinds of product expectations participants bring with them to the test, (2) how well these expectations represent those of the intended user population, and (3) how the test situation itself influences and may bias these expectations.

In an experiment studying the effects of expectations on usability ratings, participants read a positive or a negative product review or no review in the control group. The results show that expectations strongly affect usability ratings and this effect holds even in when users fail to accomplish their goals.Figure optionsDownload as PowerPoint slideResearch highlights
► Users’ expectations strongly affect usability ratings.
► Positively primed users rate a product highly even when they fail in their tasks.
► The finding calls attention to usability evaluation methods.