Citation

Law, Theng Jia and Ting, Choo Yee (2026) Enriching structured dataset via reasoning embeddings with few-shot learning for enhancing complaint classification. Discover Artificial Intelligence, 6 (1). ISSN 2731-0809

Text s44163-026-01698-y.pdf - Published Version Restricted to Repository staff only Download (2MB)

Abstract

Public complaint classification is essential for effective municipal governance, yet remains challenging due to heterogeneous structured data, severe class imbalance, and limited ability of structured features to capture underlying representation. Unlike the existing studies that relied solely on the original features for end-to-end prediction, the contribution of this study lied in transforming large language models (LLMs) generated reasoning under few-shot learning into an explicit feature engineering step, where reasoning outputs are integrated with structured data and embedded to enrich representations. Extensive experiments were conducted on a real-world municipal complaint dataset containing over 48103 instances across 180 complaint subcategories. The best-performing configuration which combined Qwen3-14B, Paraphrase-Multilingual-Mpnet-Base-V2 embeddings, and Logistic Regression, achieved an average score of 44.8%, with an accuracy of 54%, an F1-score of 23.9%, and an Area Under the Curve of 92%. Moreover, error analysis revealed low class-wise error rates alongside reduced false positive and false negative rates, indicating balanced control over both false alarms and missed detections. The Shapiro-Wilk test rejected normality (statistic 0.873, p < 0.05), while the Friedman test indicated significant differences across models (statistic 182.855, p < 0.05), further supported by non-overlapping confidence intervals with F1-score gains from 7.3–7.8% to 12–12.7%. Ablation studies highlight the critical role of each component of proposed framework, improving accuracy from 34.4 to 37.5% and F1-score from 12.5 to 14.1%. Overall, these findings demonstrated that LLM-generated reasoning effectively enhanced structured representations and predictive performance of existing Machine Learning classifiers in public complaint classification.

Item Type:	Article
Uncontrolled Keywords:	Large language model, Reasoning, Embeddings, Complaint classification, Few-shot learning, Machine learning
Subjects:	Q Science > QA Mathematics > QA71-90 Instruments and machines
Divisions:	Faculty of Computing and Informatics (FCI)
Depositing User:	Ms Suzilawati Abu Samah
Date Deposited:	29 Oct 2026 07:14
Last Modified:	29 Oct 2026 07:14
URII:	http://shdl.mmu.edu.my/id/eprint/16998

Downloads

Edit (login required)