Considerations for the use of commercial video games in controlled experiments

While commercial, off-the-shelf video games have been used often to observe cognitive and social phenomena, few studies have taken advantage of such games as research tools for conducting controlled experiments. Providing evidence that unmodified, commercial video games can be used to conduct gaming evaluations with high levels of both experimental control and ecological validity, we designed and conducted an experiment that utilized Mario Kart Wii to evaluate the effects of natural interaction on player performance. Based on our experience from that experiment, we present several concerns that require attention when using commercial video games as research tools. Providing examples of design decisions and outcomes from our experiment, we identify some of the benefits, drawbacks, and challenges of using such tools.

► We conducted a controlled experiment using Mario Kart Wii as a research tool. ► Commercial video games have inherent advantages and disadvantages as research tools. ► Commercial video games can offer both experimental control and ecological validity. ► There are challenges in using a commercial video game for a controlled experiment.