The Cropland Capture Game: good annotators versus vote aggregation methods

Baklanov, A. ORCID: https://orcid.org/0000-0003-1599-3618, Fritz, S. ORCID: https://orcid.org/0000-0003-0420-8549, Khachay, M., Nurmukhametov, O., & See, L. ORCID: https://orcid.org/0000-0002-2665-7065 (2016). The Cropland Capture Game: good annotators versus vote aggregation methods. In: Advanced Computational Methods for Knowledge Engineering - Proceedings of the 4th International Conference on Computer Science, Applied Mathematics and Applications, ICCSAMA 2016, 2-3 May, 2016, Vienna, Austria. pp. 167-180 Cham, Switzerland: Springer International Publishing. ISBN 978-3-319-38884-7 10.1007/978-3-319-38884-7_13.

Full text not available from this repository.

Abstract

The Cropland Capture game, which is a recently developed Geo-Wiki game, aims to map cultivated lands using around 17,000 satellite images from the Earth’s surface. Using a perceptual hash and blur detection algorithm, we improve the quality of the Cropland Capture game’s dataset. We then benchmark state-of-the-art algorithms for an aggregation of votes using results of well-known machine learning algorithms as a baseline. We demonstrate that volunteer-image assignment is highly irregular and only good annotators are presented (there are no spammers and malicious voters). We conjecture that the last fact is the main reason for surprisingly similar accuracy levels across all examined algorithms. Finally, we increase the estimated consistency with expert opinion from 77 to 91 % and up to 96 % if we restrict our attention to images with more than 9 votes.

Item Type:	Book Section
Uncontrolled Keywords:	Crowdsourcing; Image processing; Votes aggregation
Research Programs:	Ecosystems Services and Management (ESM)
Depositing User:	Romeo Molina
Date Deposited:	13 May 2016 11:23
Last Modified:	19 Oct 2022 05:00
URI:	https://pure.iiasa.ac.at/13204

Actions (login required)

View Item