Conflation of expert and crowd reference data to validate global binary thematic maps

Waldner, F., Schucknecht, A., Lesiv, M. ORCID:, Gallego, J., See, L. ORCID:, Pérez-Hoyos, A., d'Andrimont, R., de Maet, T., et al. (2019). Conflation of expert and crowd reference data to validate global binary thematic maps. Remote Sensing of Environment 221 235-246. 10.1016/j.rse.2018.10.039.

Full text not available from this repository.
Project: Stimulating Innovation for Global Monitoring of Agriculture and its Impact on the Environment in support of GEOGLAM (SIGMA, FP7 603719), Harnessing the power of crowdsourcing to improve land cover and land-use information (CROWDLAND, FP7 617754)


With the unprecedented availability of satellite data and the rise of global binary maps, the collection of shared reference data sets should be fostered to allow systematic product benchmarking and validation. Authoritative global reference data are generally collected by experts with regional knowledge through photo-interpretation. During the last decade, crowdsourcing has emerged as an attractive alternative for rapid and relatively cheap data collection, beckoning the increasingly relevant question: can these two data sources be combined to validate thematic maps? In this article, we compared expert and crowd data and assessed their relative agreement for cropland identification, a land cover class often reported as difficult to map. Results indicate that observations from experts and volunteers could be partially conflated provided that several consistency checks are performed. We propose that conflation, i.e., replacement and augmentation of expert observations by crowdsourced observations, should be carried out both at the sampling and data analytics levels. The latter allows to evaluate the reliability of crowdsourced observations and to decide whether they should be conflated or discarded. We demonstrate that the standard deviation of crowdsourced contributions is a simple yet robust indicator of reliability which can effectively inform conflation. Following this criterion, we found that 70% of the expert observations could be crowdsourced with little to no effect on accuracy estimates, allowing a strategic reallocation of the spared expert effort to increase the reliability of the remaining 30% at no additional cost. Finally, we provide a collection of evidence-based recommendations for future hybrid reference data collection campaigns.

Item Type: Article
Uncontrolled Keywords: Accuracy assessment; Crowdsourcing; Volunteered geographic information; Data quality; Stratified systematic sampling; Photo-interpretation
Research Programs: Ecosystems Services and Management (ESM)
Depositing User: Luke Kirwan
Date Deposited: 30 Nov 2018 09:02
Last Modified: 19 Oct 2022 05:00

Actions (login required)

View Item View Item