Expert-based versus citation-based ranking of scholarly and scientific publication channels

•We analyze the expert-based ranks in the Finnish publication forum.•We use more involved analysis techniques and explanatory variables than existing studies do.•Most ranks can be automatically constructed through reference models.•Deviating ranks can be explained by the publication activity of Finnish researchers.•Using the automatically constructed ranks would be more objective and save costs.

The Finnish publication channel quality ranking system was established in 2010. The system is expert-based, where separate panels decide and update the rankings of a set of publications channels allocated to them. The aggregated rankings have a notable role in the allocation of public resources into universities. The purpose of this article is to analyze this national ranking system. The analysis is mainly based on two publicly available databases containing the publication source information and the actual national publication activity information. Using citation-based indicators and other available information with association rule mining, decision trees, and confusion matrices, it is shown that most of the expert-based rankings can be predicted and explained using automatically constructed reference models. Publication channels, for which the Finnish expert-based rank is higher than the estimated one, are mainly characterized by higher publication activity or recent upgrade of the rank. Such findings emphasize the importance of openness of information on a ranking system, with its multifaceted evaluation.