journal article·2023-12-27Open access

Optimizing Robotic Task Sequencing and Trajectory Planning on the Basis of Deep Reinforcement Learning

Xiaoting Dong, Guangxi Wan, Peng Zeng, Chunhe Song, Shijie Cui

Biomimetics· Volume 9· Issue 1

Abstract

The robot task sequencing problem and trajectory planning problem are two important issues in the robotic optimization domain and are solved sequentially in two separate levels in traditional studies. This paradigm disregards the potential synergistic impact between the two problems, resulting in a local optimum solution. To address this problem, this paper formulates a co-optimization model that integrates the task sequencing problem and trajectory planning problem into a holistic problem, abbreviated as the robot TSTP problem. To solve the TSTP problem, we model the optimization process as a Markov decision process and propose a deep reinforcement learning (DRL)-based method to facilitate problem solving. To validate the proposed approach, multiple test cases are used to verify the feasibility of the TSTP model and the solving capability of the DRL method. The real-world experimental results demonstrate that the DRL method can achieve a 30.54% energy savings compared to the traditional evolution algorithm, and the computational time required by the proposed DRL method is much shorter than those of the evolutionary algorithms. In addition, when adopting the TSTP model, a 18.22% energy reduction can be achieved compared to using the sequential optimization model.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Reinforcement Learning in Robotics · Robotic Path Planning Algorithms · Modular Robots and Swarm Intelligence

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, China

Key Laboratory of Networked Control Systems, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, China

Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang 110169, China

University of Chinese Academy of Sciences, Beijing 100049, China

Funding

National Natural Science Foundation of China · The State Key Laboratory of Robotics of China · National Program for Funded Postdoctoral Researchers

References (37)

Show source-provided references
1. Optimization of robotic task sequencing problems by crowding evolutionary algorithms10.1109/TSMC.2021.3104862 ↗
2. Energy-saving trajectory planning for robotic high-speed milling of sculptured surfaces10.1109/TASE.2021.3063186 ↗
3. Ratiu, Mariana, and Prichici (2017). Industrial robot trajectory optimization—A review. MATEC Web Conf., 126, 02005.10.1051/matecconf/201712602005 ↗
4. Robotic task sequencing problem: A survey10.1007/s10846-015-0190-6 ↗
5. Deep Reinforcement Learning: A Brief Survey10.1109/MSP.2017.2743240 ↗
6. Zhao, L., Fan, J., Zhang, C., Shen, W., and Zhuang, J. (2023). A drl-based reactive scheduling policy for flexible job shops with random job arrivals. IEEE Trans. Autom. Sci. Eng., 1–12.10.1109/TASE.2023.3271666 ↗
7. Integrated task sequencing and path planning for robotic remote laser welding
8. Planning the sequence of tasks for harvesting robots10.1016/j.robot.2020.103591 ↗
9. Time scheduling and optimization ofindustrial robotized tasks basedon genetic algorithms10.1016/j.rcim.2014.12.003 ↗
10. Simultaneous task placement and sequence optimization in an inspection robotic cell10.1017/S0263574721000199 ↗
11. Optimal robot task scheduling based on genetic algorithms10.1016/j.rcim.2004.04.003 ↗
12. Time-optimal task scheduling for articulated manipulators in environments cluttered with obstacles10.1017/S0263574709005748 ↗
13. Task scheduling and motion planning for an industrial manipulator10.1016/j.rcim.2013.05.002 ↗
14. Gueta, L.B., Chiba, R., Ota, J., Ueyama, T., and Arai, T. (2008, January 19–23). Coordinated motion control of a robot arm and a positioning table with arrangement of multiple goals. Proceedings of the 2008 IEEE International Conference on Robotics and Automation, Pasadena, CA, USA.10.1109/ROBOT.2008.4543549 ↗
15. A novel clustering-based algorithm for solving spatially-constrained robotic task sequencing problems10.1109/TMECH.2020.3037158 ↗
16. Proseqqo: A generic solver for process planning and sequencing in industrial robotics10.1016/j.rcim.2022.102387 ↗
17. Time-optimal control of robotic manipulators along specified paths10.1177/027836498500400301 ↗
18. An evolutionary approach for time optimal trajectory planning of a robotic manipulator10.1016/S0020-0255(98)10052-X ↗
19. Evolutionary optimal trajectory planning for industrial robot with payload constraints10.1007/s00170-007-1169-7 ↗
20. An improved pso algorithm for time-optimal trajectory planning of delta robot in intelligent packaging10.1007/s00170-019-04421-7 ↗
21. Time optimal trajectory planning algorithm for robotic manipulator based on locally chaotic particle swarm optimization10.1049/cje.2021.00.373 ↗
22. Xu, Z., Wang, W., Chi, Y., Li, K., and He, L. (2023). Optimal trajectory planning for manipulators with efficiency and smoothness constraint. Electronics, 12.10.3390/electronics12132928 ↗
23. An improved whale optimization algorithm for robot time-jerk optimal trajectory planning10.1088/1742-6596/2170/1/012008 ↗
24. Optimal time-jerk trajectory planning of 6 axis welding robot using tlbo method10.1016/j.procs.2018.07.067 ↗
25. Time-optimal and smooth trajectory planning for robot manipulators10.1007/s12555-019-0703-3 ↗
26. Motion planning of robot manipulator based on improved nsga-ii10.1007/s12555-016-0693-3 ↗
27. A new approach to time-optimal trajectory planning with torque and jerk limits for robot10.1016/j.robot.2021.103744 ↗
28. A novel cartesian trajectory planning method by using triple nurbs curves for industrial robots10.1016/j.rcim.2023.102576 ↗
29. An Efficient Computation for Energy Optimization of Robot Trajectory10.1109/TIE.2021.3118367 ↗
30. Vergnano, A., Thorstensson, C., Lennartson, B., Falkman, P., Pellicciari, M., Yuan, C., Biller, S., and Leali, F. (2010, January 21–24). Embedding detailed robot energy optimization into high-level scheduling. Proceedings of the 2010 IEEE International Conference on Automation Science and Engineering, Toronto, ON, Canada.10.1109/COASE.2010.5584686 ↗
31. Energy Efficient Use of Multirobot Production Lines in the Automotive Industry: Detailed System Modeling and Optimization10.1109/TASE.2013.2285813 ↗
32. High-level scheduling of energy optimal trajectories10.1109/TASE.2012.2198816 ↗
33. Hou, R., Niu, J., Guo, Y., Ren, T., Yu, X., Han, B., and Ma, Q. (2022). A novel resolution scheme of time-energy optimal trajectory for precise acceleration controlled in dustrial robot using neural networks. Actuators, 11.10.3390/act11050130 ↗
34. Sang, W., Sun, N., Zhang, C., Qiu, Z., and Fang, Y. (2022, January 4–7). Hybrid Time-Energy Optimal Trajectory Planning for Robot Manipulators with Path and Uniform Velocity Constraints. Proceedings of the 2022 13th Asian Control Conference (ASCC), Jeju, Republic of Korea.10.23919/ASCC56756.2022.9828215 ↗
35. Shi, X., Fang, H., Pi, G., Xu, X., and Chen, H. (2018, January 17–19). Time-Energy-Jerk Dynamic Optimal Trajectory Planning for Manipulators Based on Quintic NURBS. Proceedings of the 2018 3rd International Conference on Robotics and Automation Engineering (ICRAE), Guangzhou, China.10.1109/ICRAE.2018.8586763 ↗
36. Human-level control through deep reinforcement learning10.1038/nature14236 ↗
37. Optimal robot task scheduling based on adaptive neuro-fuzzy system and genetic algorithms10.1007/s00170-020-06166-0 ↗