A Comparison Between Various Human Detectors and CNN-Based Feature Extractors for Human Activity Recognition via Aerial Captured Video Sequences


AlDahoul, Nouar and Abdul Karim, Hezerul and Md. Sabri, Aznul Qalid and Tan, Myles Joshua Toledo and Momo, Mhd. Adel and Fermin, Jamie Ledesma (2022) A Comparison Between Various Human Detectors and CNN-Based Feature Extractors for Human Activity Recognition via Aerial Captured Video Sequences. IEEE Access, 10. pp. 63532-63553. ISSN 2169-3536

[img] Text
36.pdf - Published Version
Restricted to Repository staff only

Download (3MB)


Human detection and activity recognition (HDAR) in videos plays an important role in various real-life applications. Recently, object detection methods have been used to detect humans in videos for subsequent decision-making applications. This paper aims to address the problem of human detection in aerial captured video sequences using a moving camera attached to an aerial platform with dynamical events such as varied altitudes, illumination changes, camera jitter, and variations in viewpoints, object sizes and colors. Unlike traditional datasets that have frames captured by a static ground camera with medium or large regions of humans in these frames, the UCF-ARG aerial dataset is more challenging because it contains videos with large distances between the humans in the frames and the camera. The performance of human detection methods that have been described in the literature are often degraded when input video frames are distorted by noise, blur, illumination changes, and the like. To address these limitations, the object detection methods used in this study were trained on the COCO dataset and evaluated on the publicly available UCF-ARG dataset. The comparison between these detectors was done in terms of detection accuracy. The performance evaluation considers five human actions (digging, waving, throwing, walking, and running). Experimental results demonstrated that EfficientDetD7 was able to outperform other detectors with 92.9% average accuracy in detecting all activities and various conditions including blurring, addition of Gaussian noise, lightening, and darkening. Additionally, deep pre-trained convolutional neural networks (CNNs) such as ResNet and EfficientNet were used to extract highly informative features from the detected and cropped human patches. The extracted spatial features were utilized by Long Short-Term Memory (LSTM) to consider temporal relations between features for human activity recognition (HAR). Experimental results found that the EfficientNetB7-LSTM was able to outperform existing HAR methods in terms of average accuracy (80%), and average F1 score (80%). The outcome is a robust HAR system which combines EfficientDetD7, EfficientNetB7, and LSTM for human detection and activity classification.

Item Type: Article
Uncontrolled Keywords: Convolutional neural network, Feature extraction, Cameras, Sensors, Detectors, Data mining, Activity recognition, Self-organizing feature maps
Subjects: Q Science > QA Mathematics > QA71-90 Instruments and machines > QA75.5-76.95 Electronic computers. Computer science
Divisions: Faculty of Engineering (FOE)
Depositing User: Ms Nurul Iqtiani Ahmad
Date Deposited: 29 Jul 2022 01:15
Last Modified: 29 Jul 2022 01:15
URII: http://shdl.mmu.edu.my/id/eprint/10237


Downloads per month over past year

View ItemEdit (login required)