Combining crowd consensus and user trustworthiness for managing collective tasks

• Definition of the LiquidCrowd methodological approach for managing collective tasks through crowdsourcing techniques.
• Combined use of consensus and trustworthiness techniques for task and worker management.
• Analysis of a case study about collective web-resource labeling.

In this paper, we present the LiquidCrowd methodological approach based on consensus and trustworthiness techniques for managing the execution of collective tasks. By collective task, we refer to a task in which the result depends on the consensus (i.e., degree of agreement) reached by the group of workers involved in the task execution. The more workers agree on a certain task answer, the more this answer is valid/accepted as a committed result for the task. In the paper, we provide crowdsourcing techniques for managing collective tasks based on the notion of supermajority to verify an expected degree of consensus/agreement required for considering a task as successfully completed. Moreover, we define trustworthiness techniques to measure the worker reliability/expertise in collective task execution in order to tailor the composition of a group by involving workers that are more able to express a sharable answer where needed. A case study of a collective task related to web-resource labeling is illustrated and evaluation results of the LiquidCrowd methodological approach are finally discussed.

Figure optionsDownload as PowerPoint slide