How sampling and scale limit accuracy assessment of vegetation maps: A comment on Loehle et al. (2015)

• We examined how bias and resolution of validation data impact map assessment.
• Validation plots smaller than pixel resolution underestimate map performance.
• To a lesser extent, biased validation data underestimated map performance.
• Care should be taken to ensure that sampling design does not limit map assessment.

Accuracy assessments of remote sensing products are necessary for identifying map strengths and weaknesses in scientific and management applications. However, not all accuracy assessments are created equal. Motivated by a recent study published in Forest Ecology and Management (Volume 342, pages 8–20), we explored the potential limitations of accuracy assessments related to characteristics of the field data: sampling bias and spatial resolution. The authors of the previous paper used data from variable radius plots near northern spotted owl nest sites to assess the predictive accuracy of gradient nearest neighbor (GNN) maps in portions of Oregon and Washington, USA. The field plots used for accuracy assessment (1) potentially biased the accuracy assessment toward older forests and (2) examined accuracy at finer scales than the imputation map predictions under consideration. To examine both the impacts of bias and scale in accuracy assessment, we assessed the predictive accuracy of GNN maps in western and southern Oregon. We found correlation coefficients between predicted (900 m2) and observed forest attributes for small plots (506 m2) were consistently lower than accuracy assessments using larger plots (4048 m2). Similarly, correlation coefficients based only on field plots near nest sites were lower than correlations based on all field plots. These results imply that sampling bias and small plot areas result in accuracy assessments that underestimate map predictive performance. In particular, assessing accuracy at spatial scales below the resolution of the map products are overly pessimistic (i.e., low correlation coefficients). While accuracy assessment is important, care needs to be taken to ensure that the sampling design for field data does not limit inference on map accuracy.