Citation

AlDahoul, Nouar and Tan, Myles Joshua Toledo and Tera, Raghava Reddy and Abdul Karim, Hezerul and Chee, How Lim and Mishra, Manish Kumar and Zaki, Yasir (2025) Multitasking vision language models for vehicle plate recognition with VehiclePaliGemma. Scientific Reports, 15 (1). ISSN 2045-2322

Text s41598-025-10774-9.pdf - Published Version Restricted to Repository staff only Download (2MB)

Abstract

License Plate Recognition (LPR) automates vehicle identification using cameras and computer vision. It compares captured plates against databases to detect stolen vehicles, uninsured drivers, and crime suspects. Traditionally reliant on Optical Character Recognition (OCR), LPR faces challenges like noise, blurring, weather effects, and closely spaced characters, complicating accurate recognition. Existing LPR methods still require significant improvement, especially for distorted images. To fill this gap, we propose utilizing visual language models (VLMs) such as OpenAI GPT-4o (Generative Pre-trained Transformer 4 Omni), Google Gemini 1.5, Google PaliGemma (Pathways Language and Image model + Gemma model), Meta Llama (Large Language Model Meta AI) 3.2, Anthropic Claude 3.5 Sonnet, LLaVA (Large Language and Vision Assistant), NVIDIA VILA (Visual Language), and moondream2 to recognize such unclear plates with close characters. This paper evaluates the VLM’s capability to address the aforementioned problems. Additionally, we introduce “VehiclePaliGemma”, a fine-tuned Open-sourced PaliGemma VLM designed to recognize plates under challenging conditions. We compared our proposed VehiclePaliGemma with state-of-the-art methods and other VLMs using a dataset of Malaysian license plates collected under complex conditions. The results indicate that VehiclePaliGemma achieved superior performance with an accuracy of 87.6%. Moreover, it is able to predict the car’s plate at a speed of 7 frames per second using A100-80GB GPU. Finally, we explored the multitasking capability of VehiclePaliGemma model to accurately identify plates containing multiple cars of various models and colors, with plates positioned and oriented in different directions.

Item Type:	Article
Uncontrolled Keywords:	Vehicle recognition
Subjects:	L Education > LB Theory and practice of education
Divisions:	Faculty of Artificial Intelligence & Engineering (FAIE)
Depositing User:	Ms Suzilawati Abu Samah
Date Deposited:	29 Jul 2025 04:12
Last Modified:	01 Aug 2025 02:20
URII:	http://shdl.mmu.edu.my/id/eprint/14367

Downloads

Edit (login required)