Compared to a small, supervised lab experiment, a large, unsupervised web-based experiment on a previously unknown effect has benefits that outweigh its potential costs

Research on internet-based studies has generally supported their benefits. However, that research sometimes did not directly compare internet-based to traditional delivery, often used non-experimental methods and small samples, and has not used an entirely unknown effect for the comparison to completely rule out demand characteristics. Our lab experiment (N = 180), in which participants were supervised by an experimenter, demonstrated previously unexamined effects. Both the frighteningness and disgustingness of insects made people want to kill them, and females wanted to kill the insects more than males did. There were also some interesting patterns of interaction with gender, but they were not statistically significant. However, an unsupervised, but larger, web-based experiment (N = 1301) produced the same significant main effects as the lab study, and the same patterns of interaction that had occurred at a non-significant level in the lab study occurred at a statistically significant level in the web-based study. These results add support to the finding that although web-based studies may incur risks by being unsupervised, such as some participants not being genuinely motivated to follow the instructions correctly, the risks are compensated for by the much larger sample size afforded by the web-based approach.

► A web-based and a lab study that were identical were directly compared.
► The web-based study produced essentially the same results as the lab study.
► Patterns that were not significant in the lab study were significant on the web.
► Lack of experimenter supervision did not adversely affect the web study.
► Demand characteristics were ruled out by using a previously unknown effect.