EVALUATING THE ASSESSMENT: SOURCES OF EVIDENCE FOR QUALITY ASSURANCE

High quality assessment practice is expected to yield valid and useful score-based interpretations about what the examinees know and are able to do with respect to a defined target domain. Given this assertion, the article presents a framework based on the “unified view of validity,” advanced by Cronbach and Messick over two decades ago, to assist in generating an evidence-based argument regarding the quality of a given assessment practice. The framework encompasses ten sources of evidence pertaining to six aspects: content, structure, sampling, contextual influences, score production, and utility. Each source is addressed with respect to the kinds of evidence that can be accumulated to help support the quality argument and refute rival hypotheses regarding systematic and unsystematic errors that can cause bias among the score-based interpretations. Methods and tools for obtaining the evidence are described and a sample of guiding questions for planning an assessment evaluation is presented in the concluding section.