Bayesian inference in disputed authorship: A case study of cognitive errors and a new system for decision support

Bayesian inference provides a formal framework for assessing the odds of hypotheses in light of evidence. This makes Bayesian inference applicable to a wide range of diagnostic challenges in the field of chance discovery, including the problem of disputed authorship that arises in electronic commerce, counter-terrorism and other forensic applications. For example, when two documents are so similar that one is likely to be a hoax written from the other, the question is: Which document is most likely the source and which document is most likely the hoax? Here I review a Bayesian study of disputed authorship performed by a biblical scholar, and I show that the scholar makes critical errors with respect to several issues, namely: Causal Basis, Likelihood Judgment and Conditional Dependency. The scholar’s errors are important because they have a large effect on his conclusions and because similar errors often occur when people, both experts and novices, are faced with the challenges of Bayesian inference. As a practical solution, I introduce a graphical system designed to help prevent the observed errors. I discuss how this decision support system applies more generally to any problem of Bayesian inference, and how it differs from the graphical models of Bayesian Networks.