Methods for evaluating volunteers’ contributions in a deforestation detection citizen science project

• We present methods to assess the volunteers’ performance in a citizen science project.
• Results from more than 500 volunteers are presented.
• We show that simple statistical increase the efficiency of the data collecting tasks.
• Procedures to identify malicious behavior and outliers are presented.

Today, due to the availability of free remote sensing data, efficient algorithms for image classification and increased connectivity and computing power, together with international policy initiatives, such as the United Nations Programme on Reducing Emissions from Deforestation and Forest Degradation (UN-REDD), more and more countries are investing in their own national forest monitoring schemes. However, tropical forests remain under threat worldwide. Recently, a citizen science project that enables citizens around the globe to be involved in forest monitoring tasks has been proposed, called “ForestWatchers” (www.forestwatchers.net). Its main goal is to allow volunteers (many of them with no scientific training) around the globe, with their own smartphones, tablets and notebooks, review satellite images of forested regions and confirm whether automatic assignments of forested and deforested regions are correct. Inspected images are then sent to a central database where the results are integrated to generate up-to-date deforestation maps. This approach offers a low-cost way to both strengthen the scientific infrastructure and engage members of the public in science. Here, we describe the methods developed within the scope of the ForestWatchers project to assess the volunteers’ performance. These tools have been evaluated with data of two of the project’s preliminary tasks. The first, called “BestTile”, asks volunteers to select which of several images of the same area has the least cloud cover, while in the second, called “Deforestation”, volunteers draw polygons on satellite images delimiting areas they believe have been deforested. The results from more than 500 volunteers show that using simple statistical tests, it is possible to achieve a triple goal: to increase the overall efficiency of the data collecting tasks by reducing the required number of volunteers per task, to identify malicious behavior and outliers, and to motivate volunteers to continue their contributions.