Uncertainty and the conditional variance

Statisticians have long viewed the quest for more information, for example through the acquisition of additional data, as being central to the goal of reducing uncertainty about some aspect of the world. This paper explores that objective through the variance, a common way of quantifying uncertainty. In particular, it examines the relationship between information and uncertainty. Surprisingly it shows that increasing the amount of information can in some cases increase the variance while in others it can decrease it. Which of these occurs is not explained by the seductive thesis that it depends simply on whether that uncertainty is merely aleatory–due to chance alone–or epistemic–due to lack of knowledge. Through examples it shows the relationship to be complex and a general theory elusive.