Accounting for taxonomic distance in accuracy assessment of soil class predictions

- Accounting for soil class similarity results in realistic accuracy assessments.
- Weighting can be by experts, class hierarchy, numerical taxonomy or a loss function.
- Naïve and tau statistics, accounting for chance agreement, can be weighted.
- Formulas and R functions are presented for the required computations.

Evaluating the accuracy of allocation to classes in monothetic hierarchical soil classification systems, including the World Reference Base for Soil Classification, US Soil Taxonomy, and Chinese Soil Taxonomy, is poorly-served by binomial methods (correct/incorrect allocation per evaluation observation), since some errors are more serious than others in terms of soil properties, map use, pedogenesis, and ease of mapping. Instead, evaluations should account for the taxonomic distance between classes, expressed as class similarities, giving partial credit to some incorrect allocations. These can then be used in weighted accuracy measures, either direct measures of agreement or measures that account for chance agreement, such as the tau index. Similarities can be determined in one of four ways: (1) by the expert opinion of a soil classification specialist; (2) by the distance between classes in a numerical taxonomy assessment; (3) by distance within a taxonomic hierarchy; or (4) by an error loss function. Expert opinion can be from the point of view of the map user, to assess map utility, or map producer, to assess mapping skill. Examples are given of determining similarity between a subset of Chinese Soil Taxonomy classes by expert opinion and by numerical taxonomy from soil spectra, and then using these for weighted accuracy assessment. A method for assessing the accuracy of probabilistic predictions of several classes at a location is also proposed.