Insights about large-scale online peer assessment from an analysis of an astronomy MOOC

In this work we investigate the peer grading assignments which were an integral part of the astronomy Massive Open Online Course (MOOC) (Astronomy: Exploring Time and Space) provided through Coursera from March to May 2015. Our general goal is to assess the role of peer graded assignments in such courses and how they contribute to students' learning and motivation. In order to achieve this broad goal we look at the peer grading process from multiple perspectives. We present an analysis of demographics for peer grading participants and show how they are different from the general course population. We also look at different aspects of peer grading assignments such as lengths of essays, time spent grading, number of gradings performed, final grades and percentage of relevant videos watched. We compare these distributions for different assignments and also their correlations on a level of individual learners. We show that participation in the first peer graded assignment is the best predictor of completion for the course as a whole. Moreover, learners who did well on the first peer graded assignment show better engagement and do better in the course overall. Finally, we report on validity and reliability of peer graders as compared to instructor graders and trained undergraduate graders.