Everything is illuminated: What big data can tell us about teacher commentary

• Digital portfolio tools and powerful concordance software open new possibilities.
• Large-scale data aggregated and analyzed within a very large writing program.
• Instructors did not comment excessively on surface-level errors.
• The most frequent instructor comments addressed global-level concerns.
• Instructor comment patterns such as rubber-stamping continue.

What happens to writing instructors’ feedback when they use a common rubric and an online tool to respond to student papers in a first-year composition course at a large state university in the United States? To investigate this question, we analyze the 118,611 comments instructors made when responding to 17,433 student essays. Using concordance software to quantify teachers’ use of rubric terms, we found instructors were primarily concerned with global, substantive, higher-order concerns—such as responding to students’ rhetorical situations, use of reason, and organization—rather than lower-order concerns about grammar or formatting. Given past research has determined teachers overemphasize lower-order concerns such as grammar, mechanics, and punctuation (Connors and Lunsford, 1988, Lunsford and Lunsford, 2008, Moxley and Joseph, 1989, Moxley and Joseph, 1992, Schwartz, 1984, Sommers, 1982 and Stern and Solomon, 2006), these results may suggest the possibility of a generational shift when it comes to response to student writing. Aggregating teacher commentary, student work, and peer review responses via digital tools and employing concordance software to identify big-data patterns illuminates a new assessment practice for Writing Program Administrators—the practice of Deep Assessment.