Citation

Khor, Chin Yee and Mubin, Siti Azreena and Tan, Chin Ike and Thiruchelvam, Vinesh and Neo, Tse Kian and Teoh, Gaik Kin (2025) A Multimodal Framework for Integrating Motion Capture Monitoring into ASD Screening Utilizing XR Applications. Proceedings of the ACM on Human-Computer Interaction, 9 (4). pp. 1-18. ISSN 2573-0142

Text 1.pdf - Published Version Restricted to Repository staff only Download (4MB)

Abstract

Full-body tracking (FBT) enhances embodiment and motion-based interaction in extended reality (XR), particularly for virtual reality (VR) applications such as those used for autism spectrum disorder (ASD) screening. While Meta Movement Software Development Kit (SDK) provides eye and hand tracking, its estimation-based FBT skeleton conflicts with Open Sound Control (OSC)-based body tracking data, leading to body proportion distortion and misalignment when external trackers like SlimeVR are used. To resolve this, we adopt the Virtual Motion Capture (VMC) protocol skeleton as the primary FBT framework, integrating OSC-based body tracking with OpenXR for head tracking and Meta’s SDK for eye and hand tracking. Multiple custom OSC messages are created by retrieving positional and rotational data from custom input actions, allowing real-time player headspace detection and precise avatar alignment. This approach ensures accurate skeletal proportions and real-time avatar alignment, eliminating reliance on Meta’s prediction-based body tracking. By leveraging custom OSC messages and SlimeVR’s body proportion calibration, the proposed system offers a more stable and precise FBT solution for standalone VR applications.

Item Type:	Article
Uncontrolled Keywords:	Eye Tracking, Full-Body Tracking, Multimodal Integration, Autism Spectrum Disorder, Real-Time Data Integration, User Experience Design
Subjects:	Q Science > QC Physics > QC350-467 Optics. Light > QC450-467 Spectroscopy
Divisions:	Faculty of Creative Multimedia (FCM)
Depositing User:	Ms Suzilawati Abu Samah
Date Deposited:	28 Jul 2025 06:47
Last Modified:	30 Jul 2025 07:04
URII:	http://shdl.mmu.edu.my/id/eprint/14295

Downloads

Edit (login required)