Transforming LSA space dimensions into a rubric for an automatic assessment and feedback system |
| |
| Institution: | 1. Universidad Autónoma de Madrid, Ciudad Universitaria de Cantoblanco. C/Iván Pavlov, s/n., 28049 Madrid, Spain\n;2. Universidad Nacional de Educación a Distancia, Juan del Rosal, nº 10. 28023, Spain;3. Semantia Lab, Bravo Murillo, 38. 28015, Madrid, Spain;1. Qatar Computing Research Institute, HBKU, Doha, Qatar;2. Department of Computer Science and Engineering, College of Engineering, Qatar University, Doha, Qatar;1. Polytechnic Institute of Tomar, Tomar, Portugal;2. LIAAD/INESC TEC – INESC Technology and Science, Portugal\n;3. DCC – FCUP, University of Porto, Portugal;4. HULTECH/GREYC, University of Caen Basse-Normandie, Caen, France;5. Department of Mathematics, University of Beira Interior, Covilhã, Portugal;6. Center of Mathematics, University of Beira Interior, Covilhã, Portugal;1. Université de Toulouse, Laboratoire de Génie de Production (LGP), EA 1905, ENIT-INPT, 47 Avenue d’Azereix, BP 1629, Tarbes Cedex 65016, France;2. Université de Toulouse, Faculté de droit, 2 rue du Doyen Gabriel Marty, Toulouse cedex 9 31042, France |
| |
| Abstract: | The purpose of this article is to validate, through two empirical studies, a new method for automatic evaluation of written texts, called Inbuilt Rubric, based on the Latent Semantic Analysis (LSA) technique, which constitutes an innovative and distinct turn with respect to LSA application so far. In the first empirical study, evidence of the validity of the method to identify and evaluate the conceptual axes of a text in a sample of 78 summaries by secondary school students is sought. Results show that the proposed method has a significantly higher degree of reliability than classic LSA methods of text evaluation, and displays very high sensitivity to identify which conceptual axes are included or not in each summary. A second study evaluates the method's capacity to interact and provide feedback about quality in a real online system on a sample of 924 discursive texts written by university students. Results show that students improved the quality of their written texts using this system, and also rated the experience very highly. The final conclusion is that this new method opens a very interesting way regarding the role of automatic assessors in the identification of presence/absence and quality of elaboration of relevant conceptual information in texts written by students with lower time costs than the usual LSA-based methods. |
| |
| Keywords: | |
| 本文献已被 ScienceDirect 等数据库收录！ |
|
