Citation

Tan, Kian Long and Lee, Chin Poo and Lim, Kian Ming (2023) RoBERTa-GRU: A Hybrid Deep Learning Model for Enhanced Sentiment Analysis. Applied Sciences, 13 (6). p. 3915. ISSN 2076-3417

Text applsci-13-03915.pdf - Published Version Restricted to Repository staff only Download (443kB)

Abstract

This paper proposes a novel hybrid model for sentiment analysis. The model leverages the strengths of both the Transformer model, represented by the Robustly Optimized BERT Pretraining Approach (RoBERTa), and the Recurrent Neural Network, represented by Gated Recurrent Units (GRU). The RoBERTa model provides the capability to project the texts into a discriminative embedding space through its attention mechanism, while the GRU model captures the long-range dependencies of the embedding and addresses the vanishing gradients problem. To overcome the challenge of imbalanced datasets in sentiment analysis, this paper also proposes the use of data augmentation with word embeddings by over-sampling the minority classes. This enhances the representation capacity of the model, making it more robust and accurate in handling the sentiment classification task. The proposed RoBERTa-GRU model was evaluated on three widely used sentiment analysis datasets: IMDb, Sentiment140, and Twitter US Airline Sentiment. The results show that the model achieved an accuracy of 94.63% on IMDb, 89.59% on Sentiment140, and 91.52% on Twitter US Airline Sentiment. These results demonstrate the effectiveness of the proposed RoBERTa-GRU hybrid model in sentiment analysis.

Item Type:	Article
Uncontrolled Keywords:	sentiment analysis; deep learning; Transformer; RoBERTa; GRU
Subjects:	L Education > LB Theory and practice of education > LB1060 Learning
Divisions:	Faculty of Information Science and Technology (FIST)
Depositing User:	Ms Nurul Iqtiani Ahmad
Date Deposited:	02 May 2023 07:49
Last Modified:	02 May 2023 07:49
URII:	http://shdl.mmu.edu.my/id/eprint/11392

Downloads

Edit (login required)