Understanding metacognitive confidence: Insights from judgment-of-learning justifications

•The delayed JOL paradigm was adapted to include written justifications of JOLs.•Data was collected for confidence (0-100%) and binary (yes/no) JOLs.•Justifications were analysed using a range of natural language processing techniques.•Justifications primarily referenced cue familiarity and target accessibility.•Justifications of confidence and binary JOLs did not directly map onto each other.

This study employed the delayed judgment-of-learning (JOL) paradigm to investigate the content of metacognitive judgments; after studying cue-target word-pairs, participants predicted their ability to remember targets on a future memory test (cued recognition in Experiments 1 and 2 and cued recall in Experiment 3). In Experiment 1 and the confidence JOL group of Experiment 3, participants used a commonly employed 6-point numeric confidence JOL scale (0-20-40-60-80-100%). In Experiment 2 and the binary JOL group of Experiment 3 participants first made a binary yes/no JOL prediction followed by a 3-point verbal confidence judgment (sure-maybe-guess). In all experiments, on a subset of trials, participants gave a written justification of why they gave that specific JOL response. We used natural language processing techniques (latent semantic analysis and word frequency [n-gram] analysis) to characterize the content of the written justifications and to capture what types of evidence evaluation uniquely separate one JOL response type from others. We also used a machine learning classification algorithm (support vector machine [SVM]) to quantify the extent to which any two JOL responses differed from each other. We found that: (i) participants can justify and explain their JOLs; (ii) these justifications reference cue familiarity and target accessibility and so are particularly consistent with the two-stage metacognitive model; and (iii) JOL confidence judgements do not correspond to yes/no responses in the manner typically assumed within the literature (i.e. 0-40% interpreted as no predictions).