Needle in a Haystack: Spotting and recognising micro-expressions “in the wild”


Gan, Y. S. and See, John and Khor, Huai Qian and Liu, Kun Hong and Liong, Sze Teng (2022) Needle in a Haystack: Spotting and recognising micro-expressions “in the wild”. Neurocomputing, 503. pp. 283-298. ISSN 0925-2312

[img] Text
18.pdf - Published Version
Restricted to Repository staff only

Download (2MB)


Computational research on facial micro-expressions has long focused on videos captured under constrained laboratory conditions due to the challenging elicitation process and limited samples that are publicly available. Moreover, processing micro-expressions is extremely challenging under unconstrained scenarios. This paper introduces, for the first time, a completely automatic micro-expression “spot-and-recognize” framework that is performed on in-the-wild videos, such as in poker games and political interviews. The proposed method first spots the apex frame from a video by handling head movements and unconscious actions which are typically larger in motion intensity, with alignment employed to enforce a canonical face pose. Optical flow guided features play a central role in our method: they can robustly identify the location of the apex frame, and are used to learn a shallow neural network model for emotion classification. Experimental results demonstrate the feasibility of the proposed methodology, establishing good baselines for both spotting and recognition tasks – ASR of 0.33 and F1-score of 0.6758 respectively on the MEVIEW micro-expression database. In addition, we present comprehensive qualitative and quantitative analyses to further show the effectiveness of the proposed framework, with new suggestion for an appropriate evaluation protocol. In a nutshell, this paper provides a new benchmark for apex spotting and emotion recognition in an in-the-wild setting.

Item Type: Article
Uncontrolled Keywords: Apex frame, In-the-wild, Face alignment, Micro-expression spotting, Micro-expression recognition
Subjects: Q Science > QA Mathematics > QA71-90 Instruments and machines > QA75.5-76.95 Electronic computers. Computer science
Divisions: Faculty of Computing and Informatics (FCI)
Depositing User: Ms Nurul Iqtiani Ahmad
Date Deposited: 02 Aug 2022 00:40
Last Modified: 02 Aug 2022 00:40


Downloads per month over past year

View ItemEdit (login required)