Classification of individual articles from all of science by research level

• Four research levels, characterizing the basic-to-applied spectrum of research, can be effectively modeled using words from titles and abstracts and citations.
• A multinomial logistic regression model of research levels was trained on over 11 million papers from all areas of science.
• The resulting model accurately captures variation in research level at the article level and was used to classify over 25 million documents in Scopus (1996–2011) by research level.
• Citation counts positively correlate with research level for the medical and physical sciences.

A system of four research levels, designed to classify scientific journals from most applied to most basic, was introduced by Francis Narin and colleagues in the 1970s. Research levels have been used since that time to characterize research at institutional and departmental levels. Currently, less than half of all articles published are in journals that been classified by research level. There is thus a need for the notion of research level to be extended in a way that all articles can be so classified. This article reports on a new model – trained from title and abstract words and cited references – that classifies individual articles by research level. The model covers all of science, and has been used to classify over 25 million articles from Scopus by research level. The final model and set of classified articles are further characterized.