Early career grants, performance, and careers: A study on predictive validity of grant decisions

•This paper studies predictive validity of grant decisions, including data from rejected applicants.•The grant decisions seem to have no predictive validity, as the successful applicants have no higher performance than the best performing non-successful applicants.•Career grants do help successful applicants to get higher positions.•Gender remains playing a role in academic careers.

The main rationale behind career grants is helping top talent to develop into the next generation leading scientists. Does career grant competition result in the selection of the best young talents? In this paper we investigate whether the selected applicants are indeed performing at the expected excellent level—something that is hardly investigated in the research literature.We investigate the predictive validity of grant decision-making, using a sample of 260 early career grant applications in three social science fields. We measure output and impact of the applicants about ten years after the application to find out whether the selected researchers perform ex post better than the non-successful ones. Overall, we find that predictive validity is low to moderate when comparing grantees with all non-successful applicants. Comparing grantees with the best performing non-successful applicants, predictive validity is absent. This implies that the common belief that peers in selection panels are good in recognizing outstanding talents is incorrect. We also investigate the effects of the grants on careers and show that recipients of the grants do have a better career than the non-granted applicants. This makes the observed lack of predictive validity even more problematic.