Citation

Lye Abdullah, Mohd Haris and AlDahoul, Nouar and Abdul Karim, Hezerul (2023) Fusion of Appearance and Motion Features for Daily Activity Recognition from Egocentric Perspective. Sensors, 23 (15). p. 6804. ISSN 1424-8220

Text sensors-23-06804.pdf - Published Version Restricted to Repository staff only Download (2MB)

Abstract

Vidos from a first-person or egocentric perspective offer a promising tool for recognizing various activities related to daily living. In the egocentric perspective, the video is obtained from a wearable camera, and this enables the capture of the person’s activities in a consistent viewpoint. Recognition of activity using a wearable sensor is challenging due to various reasons, such as motion blur and large variations. The existing methods are based on extracting handcrafted features from video frames to represent the contents. These features are domain-dependent, where features that are suitable for a specific dataset may not be suitable for others. In this paper, we propose a novel solution to recognize daily living activities from a pre-segmented video clip. The pre-trained convolutional neural network (CNN) model VGG16 is used to extract visual features from sampled video frames and then aggregated by the proposed pooling scheme. The proposed solution combines appearance and motion features extracted from video frames and optical flow images, respectively. The methods of mean and max spatial pooling (MMSP) and max mean temporal pyramid (TPMM) pooling are proposed to compose the final video descriptor. The feature is applied to a linear support vector machine (SVM) to recognize the type of activities observed in the video clip. The evaluation of the proposed solution was performed on three public benchmark datasets. We performed studies to show the advantage of aggregating appearance and motion features for daily activity recognition. The results show that the proposed solution is promising for recognizing activities of daily living. Compared to several methods on three public datasets, the proposed MMSP–TPMM method produces higher classification performance in terms of accuracy (90.38% with LENA dataset, 75.37% with ADL dataset, 96.08% with FPPA dataset) and average per-class precision (AP) (58.42% with ADL dataset and 96.11% with FPPA dataset).

Item Type:	Article
Uncontrolled Keywords:	Activities of daily living; convolutional neural network; egocentric vision; feature fusion; optical flow
Subjects:	T Technology > TK Electrical engineering. Electronics Nuclear engineering > TK7800-8360 Electronics > TK8300-8360 Photoelectronic devices (General)
Divisions:	Faculty of Engineering (FOE)
Depositing User:	Ms Nurul Iqtiani Ahmad
Date Deposited:	04 Sep 2023 02:31
Last Modified:	04 Sep 2023 02:31
URII:	http://shdl.mmu.edu.my/id/eprint/11644

Downloads

Edit (login required)