Determining the calibration of confidence estimation procedures for unique peptides in shotgun proteomics

The analysis of a shotgun proteomics experiment results in a list of peptide-spectrum matches (PSMs) in which each fragmentation spectrum has been matched to a peptide in a database. Subsequently, most protein inference algorithms rank peptides according to the best-scoring PSM for each peptide. However, there is disagreement in the scientific literature on the best method to assess the statistical significance of the resulting peptide identifications. Here, we use a previously described calibration protocol to evaluate the accuracy of three different peptide-level statistical confidence estimation procedures: the classical Fisher's method, and two complementary procedures that estimate significance, respectively, before and after selecting the top-scoring PSM for each spectrum. Our experiments show that the latter method, which is employed by MaxQuant and Percolator, produces the most accurate, well-calibrated results.

Graphical abstractConfidence estimates for unique peptides and peptide-spectrum matches differ. Here, we present methodology for estimating the correctness of schemes reporting peptide-level statistics.Figure optionsDownload full-size imageDownload high-quality image (263 K)Download as PowerPoint slideHighlights► Confidence estimates for unique peptides and peptide-spectrum matches (PSMs) differ. ► Methods for transferring estimates from PSMs to peptides have not been validated. ► Here, we evaluate the statistical accuracy, or calibration, of three such procedures. ► One of the procedures tested (here denoted WOTE) produces well calibrated results.