ROC analysis of lineups does not measure underlying discriminability and has limited value

Some researchers have been arguing that eyewitness identification data from lineups should be analyzed using Receiver Operating Characteristic (ROC) analysis because it purportedly measures underlying discriminability. But ROC analysis, which was designed for 2 × 2 tasks, does not fit the 3 × 2 structure of lineups. Accordingly, ROC proponents force lineup data into a 2 × 2 structure by treating false-positive identifications of lineup fillers as though they were rejections. Using data from lineups versus showups, we illustrate how this approach misfires as a measure of underlying discriminability. Moreover, treating false-positive identifications of fillers as if they were rejections hides one of the most important phenomena in eyewitness lineups, namely filler siphoning. Filler siphoning reduces the risk of mistaken identification by drawing false-positive identifications away from the innocent suspect and onto lineup fillers. We show that ROC analysis confuses filler siphoning with an improvement in underlying discriminability, thereby fostering misleading theoretical conclusions about how lineups work.