Research on trajectory optimization and adaptive control of manipulator based on deep reinforcement learning

Abstract

In this paper, the trajectory optimization and adaptive control method of manipulator based on deep reinforcement learning (DRL) is studied, aiming at solving the contradiction between real-time, safety and adaptability of manipulator in complex dynamic environment. Aiming at the limitations of traditional control methods under unstructured disturbances, a hierarchical control architecture is proposed, which includes a high-level trajectory optimization layer and a low-level adaptive control layer, and integrates the safety RL mechanism. High-level uses time-series convolutional network (TCN) combined with attention mechanism to process time-series environmental information and generate trajectory parameters that meet security constraints; The parameter adaptive module of sliding mode control is designed at the lower level, and the stability is guaranteed by Lyapunov function. Experiments were carried out on a 7-degree-of-freedom Franka Emika manipulator platform. The results show that the proposed method has the shortest trajectory length, the lowest average tracking error and the lowest collision rate (2.1%) in dynamic scenes, the fastest planning speed and the lowest energy consumption, and is significantly superior to traditional model predictive control (MPC), DRL without safety constraints and fixed sliding mode control methods. The ablation experiment further verified the contribution of each module to the system performance, and the integrated performance of the complete system was the best, achieving the lowest tracking error, collision rate and reasonable planning time.