Effect of evaluators’ cognitive style on heuristic evaluation: Field dependent and field independent evaluators

Heuristic evaluation is a widely used usability evaluation method [Rosenbaum et al., 2000. A toolkit for strategic usability: results from workshops, panels, and surveys. In: Little, R., Nigay, L. (Eds.), In: Proceedings of ACM CHI 2000 Conference, New York, pp. 337–344]. But it suffers from large variability in the evaluation results due to differences among evaluators [Nielsen, 1993. Usability Engineering. Academic Press, Boston, MA]. The evaluation performance of evaluators with two types of cognitive styles—ten field independent (FI) subjects and ten field dependent (FD) subjects were compared. The results indicated that the FI subjects produced evaluation results with significantly higher thoroughness (t18=3.49, p=0.0026), validity (t18=4.26, p=0.0005), effectiveness (t18=5.14, p=0.0001), and sensitivity (t18=3.16, p=0.005) than the FD subjects. When assessing their own evaluation experiences, the FI subjects felt it was easier to find usability problems than the FD subjects (t18=2.10, p=0.049), but the FD subjects felt more guided during the evaluation than the FI subjects (t18=2.28, p=0.035).