Exploring the Assessment of Summaries: Using Latent Semantic Analysis to Grade Summaries Written by Spanish Students

In this study we propose an integrated method to automatically assess summaries using LSA. The method is based on a regression equation calculated with a corpus of a hundred summaries (the training sample), and is validated on a different sample of summaries (the validation sample). The equation incorporates two parameters extracted from LSA: semantic similarity and vector length. A total of 396 students drawn from four stages of education participated in the study. The summaries of a short narrative text written by each participant were evaluated on a scale of 0-10 by four human graders and the scores compared to the evaluation of the summaries using LSA. The results supported that incorporating both parameters into the method resulted more successful than the traditional cosine measure, and that LSA showed a similar level of sensitivity to the quality of the summaries produced in different academic stages as that shown by the human graders.