Observations versus assessments of personality: A five-method multi-species study reveals numerous biases in ratings and methodological limitations of standardised assessments

•Novel approach to explaining the frequent lack of replicability of findings in psychology.•Methodological differences between personality assessments and observations explored.•Behavioural observations, two assessment formats and two kinds of item interpretation.•Assessment methods cannot generate scientific quantifications and therefore contain biases.•Standardised items have not narrow standardised meanings but broad fields of meanings.

Personality assessments and observations were contrasted by applying a philosophy-of-science paradigm and a study of 49 human raters and 150 capuchin monkeys. Twenty constructs were operationalised with 146 behavioural measurements in 17 situations to study capuchins’ individual-specific behaviours and with assessments on trait-adjective and behaviour-descriptive verb items to study raters’ pertinent mental representations. Analyses of reliability, cross-method coherence, taxonomic structures and socio-demographic associations highlighted substantial biases in assessments. Deviations from observations are located in human impression formation, stereotypical biases and the findings that raters interpret standardised items differently and that assessments cannot generate scientific quantifications or capture behaviour. These issues have important implications for the interpretation of findings from assessments and provide an explanation for their frequent lack of replicability.

Graphical abstractFigure optionsDownload full-size imageDownload as PowerPoint slide