Citation

Krisnawati, Lucia Dwi and Mahastama, Aditya Wikan and Haw, Su Cheng (2025) Cross-Prompt Based Automatic Short Answer Grading System. CommIT (Communication and Information Technology) Journal, 19 (2). pp. 281-291. ISSN 1979-2484

Text View of Cross-Prompt Based Automatic Short Answer Grading System.pdf - Published Version Restricted to Repository staff only Download (5MB)

Abstract

Research on Automatic Short Answer Grading (ASAG) has shown promising results in recent years. However, several important research gaps remain. Based on the literature review, the researchers identify two critical issues. First, the majority of ASAG models are trained and tested on responses to the same prompt which raises concerns about their robustness accross different prompts. Second, many existing approaches typically treat grading task as a binary classification problem. The research aims to bridge these gaps by developing an ASAG system that closely reflects real-world assessment scenarios through multiclass classification approach and cross-prompt evaluation. It is implemented by training the proposed models on 1,505 responses across 9 prompts and testing on 175 responses from 3 distinct prompts. The grading task is addressed using regression and classification techniques, including Linear Regression, Logistic Regression, Extreme Gradient Boosting (XgBoost), Adaptive Boosting (AdaBoost), and K-Nearest Neighbors (as a baseline). The grades are categorized into five classes that are represented by grade A to E. Both manual and algorithmic data augmentation techniques, including Syntactic Minority Oversampling Technique (SMOTE), are employed to address class imbalance in the sample data. Across multiple testing scenarios, all five models demonstrate consistent performance, with Linear Regression outperforming others. During the validation process, it achieves a high accuracy of 0.93, indicating its ability to correctly classify the responses. In the testing phase, it achieves a weighted F1-Score of 0.79, a macroaveraged F1-Score of 0.75, and an RMSE of 0.45. The results suggest relatively low prediction error.

Item Type:	Article
Uncontrolled Keywords:	Automatic short answer grading (ASAG), cross-prompt
Subjects:	Q Science > QA Mathematics > QA71-90 Instruments and machines > QA75.5-76.95 Electronic computers. Computer science
Divisions:	Faculty of Computing and Informatics (FCI)
Depositing User:	Nor Afiqah Mohd Adnan
Date Deposited:	09 Dec 2025 06:17
Last Modified:	09 Dec 2025 06:17
URII:	http://shdl.mmu.edu.my/id/eprint/14993

Downloads

Edit (login required)