Abstract

One of the most important goals in Intelligent Tutoring is to create applications that can evaluate the quality of a text in a human-like manner. The aim of this study is to compare three methods of using Latent Semantic Analysis (LSA) to evaluate the quality of summaries written by students in Spanish. The sample is made up by 226 summaries written by Chilean students based on both expository and narrative texts. Each summary was first assessed by human judges in order to compare the results with the scoring provided by three different LSA methods: a) comparing the summaries with the original text divided in paragraphs, b) comparing the summaries with the text as one unit, and c) comparing the summaries with other summaries written by four human experts. Results show that comparison between each student's summary and the text as a one unit constitutes the method which most closely resembles human evaluation. Copyright © 2009, Assocation for the Advancement of ArtdicaI Intelligence (www.aaai.org). All rights reserved.