Estimating Business and Management journal quality from the 2008 Research Assessment Exercise in the UK

The 2008 Research Assessment Exercise in the UK involved the peer review of over 12,500 research outputs in Business and Management, of which 92% were journal articles. Each output was graded on a 4-point scale from “world leading” to “national” with a fifth point being unclassified. These grades were accumulated for each department to provide an overall quality profile in terms of the proportions of its outputs in each category. The assessments of individual papers were not made public but the papers submitted by each department were. This data provides a major opportunity for addressing issues of concern about the evaluation of research and the effects of journal rankings, as well as the possibility of reconstructing the judgements made by the Panel about journal quality. Given the submission details and the resulting grade profile for each department, we have used linear programming to produce the best estimate of the grades awarded to papers from each journal that had more than three entries. This provides both a grade profile for each journal and a single quality estimate. The results are shown to have good validity in comparison with other journal rankings. Apart from providing a ranking of 700 journals based on the RAE results, the paper is also able to shed light on issues such as the accuracy and coverage of the ABS ranking; the degree of selectivity of submissions; the dispersion of grades for a journal; and differences between different subject areas.

► Estimating journal quality from a large scale peer review of individual papers. ► New methodology using linear programming for estimating journal quality. ► Shows the extent to which papers within a journal may have very different qualities.