Citation

Nagappan, Sidharrth and Goh, Hui Ngo and Lim, Amy Hui Lan (2022) Hybrid Deep Neural Networks for Industrial Text Scoring. In: International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, 26-29 July 2021, Kuala Lumpur, Malaysia.

Full text not available from this repository.

Abstract

Academic scoring is mainly explored through the pedagogical fields of Automated Essay Scoring (AES) and Short Answer Scoring (SAS), but text scoring in other domains has received limited attention. This paper focuses on industrial text scoring, namely the processing and adherence checking of long annual reports based on regulatory requirements. To lay the foundations for non-academic scoring, a pioneering corpus of annual reports from companies is scraped, segmented into sections, and domain experts score relevant sections based on adherence. Subsequently, deep neural non-hierarchical attention-based LSTMs, hierarchical attention networks and longformer-based models are refined and evaluated. Since the longformer outperformed LSTM-based models, we embed it into a hybrid scoring framework that employs lexicon and named entity features, with rubric injection via word-level attention, culminating in a Kappa score of 0.9670 and 0.820 in both our corpora, respectively. Though scoring is fundamentally subjective, our proposed models show significant results when navigating thin rubric boundaries and handling adversarial responses. As our work proposes a novel industrial text scoring engine, we hope to validate our framework using more official documentation based on a broader range of regulatory practices.

Item Type:	Conference or Workshop Item (Paper)
Uncontrolled Keywords:	Natural Language Processing (NLP), Deep Learning (DL), Longformer
Subjects:	Q Science > QP Physiology > QP351 Neurophysiology and Neuropsychology
Divisions:	Faculty of Computing and Informatics (FCI)
Depositing User:	Ms Nurul Iqtiani Ahmad
Date Deposited:	07 Oct 2022 01:13
Last Modified:	07 Oct 2022 01:13
URII:	http://shdl.mmu.edu.my/id/eprint/10483

Downloads

Edit (login required)