Unleashing expert judgment in assessment

Assessment evaluates accumulated knowledge and its limits. It informs and ideally empowers decisions and actions on complex, contested issues with persistent uncertainties. Applying rigorous expert judgment is an important dimension of assessment. Here we evaluate advances and challenges in approaches to expert judgment in the Intergovernmental Panel on Climate Change's Fifth Assessment Report (IPCC AR5). We find that revised guidance for author teams improved the development of balanced judgments on scientific evidence across disciplines. In particular, expert judgments underpinning conclusions are more extensively, transparently, and consistently communicated: degree-of-certainty terms are more abundant in AR5 policymaker summaries; wider ranges of possible outcomes are presented with greater inclusion of lower-certainty, decision-relevant findings; and expert judgments supporting conclusions are more comparable across working groups. But challenges in developing and communicating assessment conclusions persist, especially for findings with substantial uncertainties and for subjective aspects of judgments. Based on our evaluations and AR5 lessons learned, we propose a simpler, more rigorous framework for developing and communicating expert judgments in environmental assessment. We also describe practices for reducing expert-judgment biases, for advancing integration of evidence and expert judgment, and for addressing subjective dimensions of expert opinion directly and proactively.