A sensitivity analysis of research institutions’ productivity rankings to the time of citation observation

One of the critical issues in bibliometric research assessments is the time required to achieve maturity in citations. Citation counts can be considered a reliable proxy of the real impact of a work only if they are observed after sufficient time has passed from publication date. In the present work the authors investigate the effect of varying the time of citation observation on accuracy of productivity rankings for research institutions. Research productivity measures are calculated for all Italian universities active in the hard sciences in the 2001–2003 period, by individual field and discipline, with the time of the citation observation varying from 2004 to 2008. The objective is to support policy-makers in choosing a citation window that optimizes the tradeoff between accuracy of rankings and timeliness of the exercise.

► The analysis is based on all Italian academic papers indexed in WoS 2001–2003.
► University rankings rise from field-standardized productivity.
► Rankings are calculated varying the time of citation observation from 2004 to 2008.
► We notice substantial stability in performance rankings as citation window varies.
► Tradeoff between accuracy and unwanted delay in carrying out evaluation is much less dramatic than expected.