Assessing quality of volunteer crowdsourcing contributions: lessons from the Cropland Capture game

11287 19 61177 4 168474 document 61177 Assessing quality of volunteer crowdsourcing contributions.pdf application/pdf e8950e5f6b5d914375619b301e071ef3 MD5 592065 2016-06-07 15:06:46 https://pure.iiasa.ac.at/id/eprint/11287/1/Assessing%20quality%20of%20volunteer%20crowdsourcing%20contributions.pdf 11287 1 1 application/pdf text en public cc_by_nc

Assessing quality of volunteer crowdsourcing contributions.pdf

accepted 61178 2 168477 document 61178 lightbox.jpg image/png 04401d9a9a510112cd65ef1342b9d8f3 MD5 21137 2016-06-07 15:16:57 https://pure.iiasa.ac.at/id/eprint/11287/2/lightbox.jpg 11287 2 2 image/png other Thumbnails conversion from text to thumbnail_lightbox en public

lightbox.jpg

preview.jpg

medium.jpg

small.jpg

indexcodes.txt

http://eprints.org/relation/isVersionOf https://pure.iiasa.ac.at/id/document/61177 http://eprints.org/relation/isVolatileVersionOf https://pure.iiasa.ac.at/id/document/61177 http://eprints.org/relation/isIndexCodesVersionOf https://pure.iiasa.ac.at/id/document/61177 archive 351 disk0/00/01/12/87 2016-01-15 08:52:29 2024-01-04 14:05:06 2016-01-15 08:52:29 article show Salk C.F. 8556 Sturn T. 2051 See L. 8571 0000-0002-2665-7065 Fritz S. 8130 0000-0003-0420-8549 Assessing quality of volunteer crowdsourcing contributions: lessons from the Cropland Capture game pub iis_env prog_esm prog_pds crowdsourcing; volunteered geographic information; cropland; data quality; image classification; Geo-Wiki Volunteered geographic information (VGI) is the assembly of spatial information based on public input. While VGI has proliferated in recent years, assessing the quality of volunteer-contributed data has proven challenging, leading some to question the efficiency of such programs. In this paper, we compare several quality metrics for individual volunteers' contributions. the date were the product of the 'Cropland Capture' game, in which several thousand volunteers assessed 165,000 images for the presence of cropland over the course of 6 months. We compared agreement between volunteer ratings and an image's majority classification with volunteer self-agreement on repeated images and expert evaluations. We also examined the impact of experience and learning on performance. Volunteer self-agreement was nearly always higher than agreement with majority classifications, and much greater than agreement with expert validations although these metrics were all positively correlated. Volunteer quality showed a broad trend toward improvement with experience, but the highest accuracies were achieved by a handful of moderately active contributors, not the most active volunteers. Our results emphasize the importance of a universal set of expert-validated tasks as a gold standard for evaluating VGI quality. 2016 published Taylor & Francis 10.1080/17538947.2015.1039609 XJ-15-335 [doi:10.1080/17538947.2015.1039609] 262 301 276 98 public International Journal of Digital Earth 9 4 310-426 TRUE 1753-8947 Harnessing the power of crowdsourcing to improve land cover and land-use information (CROWDLAND, FP7 617754) Geo-Wiki FALSE yes info:eu-repo/grantAgreement/EC/FP7/617754/EU/Harnessing the power of crowdsourcing to improve land cover and land-use information/CROWDLAND info:eu-repo/semantics/article info:eu-repo/semantics/openAccess