Observer ratings: Validity and value as a tool for animal welfare research

Ratings by human observers have long been used by animal scientists and veterinarians to assess certain physical traits (e.g. body fat), and can also be applied to the assessment of behaviour and a variety of welfare-relevant variables (e.g. pain responsiveness, alopecia/barbering). Observer ratings offer a myriad of advantages, not just practical (e.g. money-saving) but also scientific: they can be used to integrate multimodal information across time and situations, and for constructs that are otherwise very difficult to assess (e.g. nest quality). Because observer ratings involve subjective judgements, some researchers may question whether they can be trusted to reflect reality in an unbiased manner. In this paper, I present evidence from a range of zoo, laboratory and farm animal studies demonstrating that observer ratings can be both reliable and valid. They have been shown to predict important biological phenomena such as reproductive success in rhinoceroses and cheetahs. Biases are indeed a risk, particularly when the ratings could reflect on the observer's own care of the animals or on their institution; however, this risk can be minimized through careful experimental design, including blinding and careful phrasing of the questions the observers need to answer. I review the steps involved in validating an observer rating scheme, and also discuss both study design issues (e.g. selecting terms to be rated and appropriate observers) and the statistical issues some schemes may raise (e.g. ordinal data are not truly normal).