From paper to web: Mode equivalence of the ARHQ and NEO-FFI

• The NEO-FFI and ARHQ in Icelandic were tested for measurement equivalence between paper and web modes.
• A randomized crossover design was applied to control for confounding by selection.
• Psychometric properties were equivalent for both measures in the respective modes.
• Response style bias may incur mode effects not detected by traditional psychometric methods.
• Both NEO-FFI and ARHQ in Icelandic can be administered in paper and web mode.

Mixed-mode questionnaires are increasingly used in research. Psychological measures, developed for paper-and-pencil (paper) administration require measurement equivalence testing when administered in an alternative mode. Here, Icelandic translations of the NEO-FFI personality measure and Adult Reading History Questionnaire (ARHQ) were tested for equivalence of measurement and data quality between paper and web mode. Perceived sensitivity of data and preference for survey mode were also assessed. One hundred adults were recruited to answer both modes in a randomized, crossover design. Eighty-eight participants completed both administrations with an average of 63.8 days (SD = 2.2) between them. Within-subjects comparisons of means between modes demonstrated measurement equivalence for both measures. However, differing invalidity coefficients by mode observed by multi-trait multi method (MTMM) analysis suggested systematic effects not captured by traditional psychometric evaluation. Of note was a greater tendency for acquiescence responding to the NEO-FFI observed in web mode. Neither personality traits nor ARHQ reading difficulty scores were associated with preference for survey mode. However, 36% of participants who considered their personality data moderately to highly sensitive scored higher in neuroticism and lower in agreeableness compared to those rating their personality data of low sensitivity.While the Icelandic NEO-FFI and ARHQ have demonstrated measurement equivalence in paper and web mode, mode equivalence of psychometric measures may require ongoing evaluation as perceptions of web privacy continue to evolve.