Unproctored internet-based device-type effects on test scores: The role of working memory

Despite the burgeoning use of unproctored Internet-based testing (UIT) in employment-related assessments, there have been limited theoretical and empirical advancements to explain the effects of technology on test and assessment outcomes. Indeed, this issue is potentially germane to all researchers and practitioners who use UIT assessments in their research and practice. To address this gap, Arthur, Keiser, and Doverspike (2017) advanced the Structural Characteristics/Information Processing (SCIP) framework as a means for psychologically conceptualizing the effect of UIT device-types on test scores. The present study examined the working memory (WM) propositions advanced by the SCIP framework. Participants were randomly assigned to a desktop computer or smartphone condition to complete a general mental ability (GMA), and personality (i.e., agreeableness, conscientiousness) measure on either a desktop computer (n = 174) or smartphone (n = 173). All participants also completed a WM measure on a desktop computer. The results provide initial support for some of the SCIP framework propositions in that as hypothesized, the WM/GMA, and WM/completion time relationships were stronger for assessments completed on smartphones, compared to desktop computers; in contrast, the WM/personality relationships were weak and did not generally differ across device types. Consequently, this study offers an initial empirical test of the SCIP framework; however, further research is needed to examine additional aspects of the framework, including the role of the other information processing variables (i.e., perceptual speed and visual acuity, psychomotor ability, and selective attention) advanced by the framework.