Evaluating prior scholarship in literature reviews of research articles: A comparative study of practices in two research paradigms

Evaluations of prior scholarship play a crucial role in the literature review (LR) of a research article by showing how the boundary of an area of inquiry can be further advanced by the writer’s work. Yet, many inexperienced writers find evaluating others’ work a major challenge. Although the task has received some attention in research and writing manuals, advice given tends to be rather general, often with little distinction made of practices followed in different paradigms. This gap could have resulted from the scarcity of empirical work in this regard. Motivated by both these gaps, this study examines how researchers in two paradigms evaluate others’ work in LRs. LRs of 80 articles were collected from four journals of Information Systems. Forty of the articles were drawn from two journals that follow a strong behavioural science research paradigm while the other 40 were from two journals that show a strong design science research paradigm. Evaluations in the LRs were analyzed based on a conceptual framework developed from the CARS model (Swales, 1990) and the work on academic evaluations in Hunston, 1993a and Hunston, 1993b and Hunston and Thompson (2000). The analysis aims to characterize and compare (a) the major types of evaluation acts performed by writers in the two research domains, (b) the entities that they evaluate and (c) the value parameters they employ. Results show marked cross-domain differences in the three areas. This article will discuss how the findings and the integrated conceptual framework can inform teaching of evaluations in LRs.

► Evaluations of prior scholarship in 80 articles of two domains of Information Systems were compared.
► The two domains are design science research (DSR) and behavioural science research (BSR).
► In the DSR articles, design and performance of prior IT artefacts are evaluated.
► In the BSR articles, validity of prior theoretical models of IT-related behaviours is evaluated.
► The analysis was informed by Hunston’s work on evaluations and Swales’ CARS model.