On some flaws of university rankings: The example of the SCImago report

Using France as our main example, we show there is a much scope for improving the SCImago ranking. We detect problems of nomenclature, double affiliation, aggregation and of bias toward large public-funded research organizations. The output per scholar is more important than the output per organization. The examples we cite suggest that only detailed knowledge of the situation can help in addressing the issue and take us beyond any automatic reading of the metadata.

► University rankings are criticized through the example of the SCImago report. ► We find problems of nomenclature, double affiliation, and aggregation. ► Only detailed knowledge of the situation can avoid double counting. ► The output per scholar is more important than the output of each institution.