Exploring beyond simple demographic variables: Differences between traditional laboratory samples and crowdsourced online samples on the Big Five personality traits

Amazon's Mechanical Turk (MTurk), a popular crowdsourcing website, is increasingly being utilized by researchers to obtain psychological data. This transition has prompted evaluation of sourcing costs, psychometric properties, and motivations of participants. However, research is limited comparing traditional and crowdsourced participants on personality measures. Therefore, in the current study laboratory participants (drawn from three universities) and MTurk workers completed the Big Five inventory and provided demographic information using web-based surveys. Controlling for age and gender, laboratory participants were significantly lower in Openness (d̂ = 0.26), and higher in Extraversion (d̂ = 0.37), Agreeableness (d̂ = 0.15), and Neuroticism (d̂ = 0.05) than MTurk participants. However, pairwise comparisons among individual sites revealed there were means above and below that for MTurk participants for Openness and Conscientiousness. Given these differences, researchers are encouraged to consider how such personality characteristics may influence the outcomes of their research when designing and conducting psychological studies that use crowdsourcing techniques to recruit participants.