How to Increase the Accuracy of Crowdsourcing Campaigns?

Nurmukhametov, O., Baklanov, A. ORCID: https://orcid.org/0000-0003-1599-3618, Fritz, S. ORCID: https://orcid.org/0000-0003-0420-8549, Khachay, M., Salk, C., See, L. ORCID: https://orcid.org/0000-0002-2665-7065, & Shchepashchenko, D. ORCID: https://orcid.org/0000-0002-7814-4990 (2015). How to Increase the Accuracy of Crowdsourcing Campaigns? In: Systems Analysis 2015 - A Conference in Celebration of Howard Raiffa, 11 -13 November, 2015, Laxenburg, Austria.

Preview

Text (Conference Poster) How to Increase the Accuracy of Crowdsourcing Campaigns.pdf - Published Version Available under License Creative Commons Attribution Non-commercial Share Alike. Download (6MB) | Preview

Abstract

Crowdsourcing is a new approach to performing tasks, with a group of volunteers rather than experts. For example, the Geo-Wiki project [1] aims to improve the global land-cover map by crowdsourcing for image recognition. Though crowdsourcing gives a simple way to perform tasks that are hard to automate, analysis of data received from non-experts is a challenging problem that requires a holistic approach. Here we study in detail the dataset of the Cropland Capture game (part of Geo-Wiki project) to increase the accuracy of campaign’s results. Using this analysis, we developed a methodology for a generic type of crowdsourcing campaign similar to the Cropland Capture game. The proposed methodology relies on computer vision and machine learning techniques. Using the Cropland Capture dataset we showed that our methodology increases agreement between aggregated volunteers’ votes and experts’ decisions from 77% to 86%.

[1] Fritz, Steffen, et al. “Geo-Wiki. Org: The use of crowdsourcing to improve global land cover.” Remote Sensing 1.3 (2009): 345-354.

Actions (login required)

View Item