Citation

Huang, Hanyi and Arogbonlo, Adetokunbo and Yu, Samson and Kwek, Lee Chung (2024) Reinforcement Learning Integrated Active Force Control for Five-link Biped Robots. In: 2024 IEEE International Systems Conference (SysCon), 15-18 Apr 2024, Montreal, QC, Canada.

Text 2.pdf - Published Version Restricted to Repository staff only Download (1MB)

Abstract

In this paper, we present a novel control model that combines reinforcement learning (RL) and active force control (AFC) for the trajectory tracking of biped robots. The AFC-RL controller is crafted to harness the robustness of RL and the simplicity of AFC in trajectory tracking. Implemented on a 5-link biped robot, the controller aims to enhance trajectory tracking and disturbance rejection capabilities. To estimate the inertia matrix of AFC in real-time, an RL module is trained using feedback tracking errors from the biped robot. We employ a deep deterministic policy gradient algorithm coupled with two reward schemes—one based on duration and the other on trajectory-following accuracy. These reward schemes synergistically enable RL to estimate the inertia matrix of AFC while minimizing errors. With an accurate inertia matrix estimation, the AFC-RL output torques are controlled to achieve precise trajectory tracking for the biped robot. For performance evaluation, we developed a five-link biped robot model in MATLAB/Simulink and conducted simulations with various routes and external disturbances. Our AFC-RL controller is compared with an existing AFC employing an iterative learning (IL) scheme from the literature. Across all scenarios, AFC-RL consistently outperforms AFC-IL in terms of average tracking error (ATE). Notably, in the case of varying sinusoid disturbances, AFC-RL surpasses AFC-IL by 2.25% in ATE. The evaluation results affirm the effectiveness of the proposed AFC-RL model for trajectory tracking and disturbance rejection in controlling biped robots.

Item Type:	Conference or Workshop Item (Paper)
Uncontrolled Keywords:	Active Force Control, Reinforcement Learning, Biped Robot, Deep Deterministic Policy Gradient, Inertia Matrix
Subjects:	T Technology > TK Electrical engineering. Electronics Nuclear engineering > TK7800-8360 Electronics
Divisions:	Faculty of Engineering and Technology (FET)
Depositing User:	Ms Nurul Iqtiani Ahmad
Date Deposited:	01 Aug 2024 06:39
Last Modified:	01 Aug 2024 06:39
URII:	http://shdl.mmu.edu.my/id/eprint/12729

Downloads

Edit (login required)