Citation

Tan, Yi Wen and Woh, Jun Meng and Yong, Ee Sin and Pang, Wai Leong and Goh, Hui Hwang and Chan, Kah Yoong and Happonen, Ari (2026) Real-Time Human Interaction Mimicry Teleoperation in Unitree G1 Edu Humanoid Robots Using the RGB Sensor. Computers, Materials & Continua, 89 (1). pp. 1-10. ISSN 1546-2226

Text TSP_CMC_83418.pdf - Published Version Restricted to Repository staff only Download (10MB)

Abstract

Along with the rapid advancement of Artificial Intelligence (AI), humanoid robots are foreseen to have great potential in the service industry, where human interaction is unavoidable. However, current systems face significant hurdles, including Field of View (FoV) problems, markerless real-time mimicry capabilities for humanoid’s fingers and arms. This study addresses these hurdles by developing an integrated hardware and software pipeline for the Unitree G1 Edu humanoid robot. A custom 3D-printed helmet and stabiliser interface were designed using FreeCAD and fabricated to house an external Orbbec Gemini 2 RGB-D sensor, optimising the FoV for frontal human-robot interaction by compensating for the robot’s height and its downward-facing onboard sensors. Both finger control and arm control use MediaPipe Holistic to extract the coordinates and perform kinematics calculation to obtain the angle commands for each joint, which are then passed through a first-order Exponential Moving Average (EMA) Filter for jitter avoidance. The safety mechanism was verified in MuJoCo simulator, followed by physical implementation on G1 humanoid. Under single-subject test in controlled environment, MediaPipe shows perception robustness of almost 100% tracking reliability for arm-level mimicry across various distances, lighting and occlusion conditions, but experiences serious performance degradation under severe backlighting. Low Root Mean Square Errors (RMSE) of 5.3°–5.6° for shoulder elevation and 8.7°–8.8° for elbow flexion, demonstrating high retargeting reproducibility. Evaluation of the jitter suppression stage indicates that the EMA filter successfully attenuated high-frequency pose estimation noise, yielding a 40.3%–63.4% jitter reduction in shoulder channels and a 34.2%–38.5% reduction in elbow channels while maintaining an optimal balance with tracking responsiveness. Deployment of the algorithm onto G1 Edu humanoid successfully verified the feasibility of the developed algorithm. Combining the algorithm with the customised helmet, this system provides a new framework for human-mimicking robotics development. Future work will focus on sensor fusion, multi-subject and dynamic environment test to better synchronise robot movement with real-world interaction scenario, which are vital for moving the technology from the lab to public service applications.

Item Type:	Article
Uncontrolled Keywords:	Machine learning, humanoid, field of view, real-time processing, artificial intelligence, dynamic environment, industry 5.0, mimicry capability, RGB-D, human-robot interaction
Subjects:	Q Science > QA Mathematics > QA71-90 Instruments and machines
Divisions:	Faculty of Artificial Intelligence & Engineering (FAIE)
Depositing User:	Ms Suzilawati Abu Samah
Date Deposited:	04 Sep 2026 03:27
Last Modified:	04 Sep 2026 03:27
URII:	http://shdl.mmu.edu.my/id/eprint/16699

Downloads

Edit (login required)