Citation

Eirina, Aina and Packier Mohammad, Nathar Shah (2025) A Hybrid RAG-Based Chatbot for University Customer Service: Combining Local LLM with Semantic Retrieval for Privacy and Real-Time Performance. Journal of Logistics, Informatics and Service Science, 12 (8). pp. 58-72. ISSN 2409-2665

Text Scopus - Document Details.pdf - Published Version Restricted to Repository staff only Download (212kB)

Abstract

The improvement of digital transformation in higher education institutions has resulted in a significant increase in the volume and complexity of student inquiries, which consistently overwhelm traditional customer service systems. This challenge creates a critical vulnerability in the university service value chain, leading to delays and inconsistent responses. While Retrieval-Augmented Generation (RAG) architectures effectively address issues of hallucination and relevance, common cloud-based RAG solutions introduce substantial risks related to data privacy, security, and institutional governance, particularly concerning sensitive student data governed by regulations such as PDPA, FERPA and GDPR. This study addresses this gap by introducing a robust, locally deployable hybrid RAG framework. The system integrates Ollama’s local Large Language Model (LLM) inference (Qwen-7B) with ChromaDB’s semantic vector search to provide accurate, real-time, and inherently privacy-conscious responses to domain-specific inquiries. Evaluation on a carefully curated, albeit constrained, dataset of 30 university inquiries demonstrates the hybrid system's effectiveness. The system significantly outperforms a generative-only baseline across key metrics, achieving a 25.0% higher BLEU score (0.75) and a 16.7% reduction in average response latency (150 ms) compared to the baseline (180 ms). The deployment of this architecture validates the feasibility of achieving high-performance AI integration while maintaining strict institutional control over sensitive data assets.

Item Type:	Article
Uncontrolled Keywords:	Data privacy, generative AI
Subjects:	Q Science > QA Mathematics > QA71-90 Instruments and machines > QA75.5-76.95 Electronic computers. Computer science
Divisions:	Faculty of Computing and Informatics (FCI)
Depositing User:	Ms Rosnani Abd Wahab
Date Deposited:	22 Dec 2025 01:17
Last Modified:	24 Dec 2025 07:06
URII:	http://shdl.mmu.edu.my/id/eprint/15077

Downloads

Edit (login required)