Evolutionary approach for crowdsourcing quality control

Crowdsourcing is widely used for solving simple tasks (e.g. tagging images) and recently, some researchers (Kittur et al., 2011 [9] and Kulkarni et al., 2012 [10]) propose new crowdsourcing models to handle complex tasks (e.g. article writing). In both type of crowdsourcing models (for simple and complex tasks), voting is a technique that is widely used for quality control [9]. For example, 5 workers are asked to write 5 outlines for an article, and another 5 workers are asked to vote for the best outline among the 5 outlines. However, we argue that voting is actually a technique that selects a high quality answer from a set of answers. It does not directly enhance answer quality. In this paper, we propose a new quality control approach for crowdsourcing that can incrementally improve answer quality. The new approach is based upon two principles - evolutionary computing and slow intelligence, which help the crowdsourcing system to propagate knowledge among workers and incrementally improve the answer quality. We perform explicitly 2 experimental case studies to show the effectiveness of the new approach. The case study results show that the new approach can incrementally improve answer quality and produce high quality answers.