The research group led by Li Guangliang of the Department of Information Science
has made significant progress in the field of artificial intelligence and robotics.
Publishing time：2022-12-20
Recently, the research group led by Li Guangliang from the Electronic Engineering School of the Faculty of Information Science and Engineering at China Ocean University has made significant progress in the fields of artificial intelligence and robotics. Their paper "Transferring Policy of Deep Reinforcement Learning from Simulation to Reality for Robotics" has been accepted and published by the international top academic journal Nature's sub-journal "Nature Machine Intelligence". "Nature Machine Intelligence" focuses on the most influential scientific research achievements in the field of artificial intelligence, machine learning, and robotics, with strict review standards, publishing approximately 100 lengthy papers each year.
Intelligence and autonomy are the current forefronts, hotspots, and difficult issues in robotics research around the world. As a new method of artificial intelligence, deep reinforcement learning imitates human trial-and-error learning mechanisms, allowing intelligent agents to autonomously learn and optimize control strategies through interaction with the external environment and experiences of trial and error, thereby improving robot intelligence and autonomy. In recent years, deep reinforcement learning has achieved great success in many fields, showing enormous potential in the field of autonomous robot control. However, on the one hand, artificial intelligence techniques such as deep reinforcement learning may pose risks, loss of control, or even harm to robots. On the other hand, the learning efficiency and safety of robots have become bottlenecks for the application of deep reinforcement learning to physical robot control.
To address the above problems, the research group creatively proposed involving humans in the autonomous learning process of robots to improve the learning efficiency and control system confidence of robots. This approach has been successfully applied to the Socially Intelligent Robotics Consortium's Haru (a social service robot developed jointly by the research group, Honda R&D Co., Ltd., MIT, Imperial College London, and other researchers), single autonomous underwater vehicles (AUVs), and multi-AUV formation control. The relevant research results have been accepted and published by top international robotic and intelligent agent academic conferences, including ICRA (2021), IROS (2022, 2021), AAMAS (2021), and top international journals in the fields of ocean engineering such as "Ocean Engineering" (2022a, 2022b, 2021) and top journal in the field of intelligent agents and multi-agent systems - "Journal of AAMAS" (2020). ICRA and IROS are the largest and most influential robotics conferences worldwide, while Ocean Engineering is a top academic journal in the field of ocean engineering.
Although introducing human supervision and feedback can greatly improve the learning efficiency of robots, the long execution time of actions may still require a long time for physical robots to learn control strategies and may cause safety issues. Inspired by the transfer learning mechanisms in animal and human brains, the research group conducted in-depth research on transfer reinforcement learning methods from simulation to reality, proposing to further improve the learning efficiency of robots and reduce the risks of executing control strategies by fully utilizing simulation data.
In their paper published in "Nature Machine Intelligence," the research group conducted a thorough comparative analysis of the latest progress, common methods, principles, application backgrounds, and development status of transfer reinforcement learning methods from simulation to reality. They proposed applying ideas from various methods, such as system identification, inverse reinforcement learning, interactive reinforcement learning, robust reinforcement learning, offline and online reinforcement learning, to the field of transfer reinforcement learning from simulation to reality. This will help to investigate more efficient transfer reinforcement learning methods and has important practical significance for accelerating the implementation of deep reinforcement learning in intelligent robot control.
For example, to solve the problem of mismatch between observation states or dynamic models in virtual and real environments, domain randomization provides rich experience for training robot control strategies by randomly adding biases and noise to visual images and dynamic parameters, achieving one-time transfer from virtual policies to real environments.
Principles and Applications of Domain Randomization.
Associate Professor Li Guangliang is the sole corresponding author of the paper, while graduate students Ju Hao and Jun Rongshun are co-first authors. China Ocean University is the first author and corresponding author unit. Li Guangliang's research group focuses on deep reinforcement learning and its applications in robotics. In recent years, they have made a series of research achievements in fields such as human-in-the-loop reinforcement learning, transfer reinforcement learning from simulation to reality, and multi-agent reinforcement learning, which has had a significant impact on relevant fields at home and abroad. The related research work was supported by the National Natural Science Foundation of China, Shandong Provincial Natural Science Foundation, and Honda Research Institute Japan.
