Towards Closing the Sim-to-Real Gap in Collaborative Multi-Robot Deep Reinforcement Learning

by   Wenshuai Zhao, et al.

Current research directions in deep reinforcement learning include bridging the simulation-reality gap, improving sample efficiency of experiences in distributed multi-agent reinforcement learning, together with the development of robust methods against adversarial agents in distributed learning, among many others. In this work, we are particularly interested in analyzing how multi-agent reinforcement learning can bridge the gap to reality in distributed multi-robot systems where the operation of the different robots is not necessarily homogeneous. These variations can happen due to sensing mismatches, inherent errors in terms of calibration of the mechanical joints, or simple differences in accuracy. While our results are simulation-based, we introduce the effect of sensing, calibration, and accuracy mismatches in distributed reinforcement learning with proximal policy optimization (PPO). We discuss on how both the different types of perturbances and how the number of agents experiencing those perturbances affect the collaborative learning effort. The simulations are carried out using a Kuka arm model in the Bullet physics engine. This is, to the best of our knowledge, the first work exploring the limitations of PPO in multi-robot systems when considering that different robots might be exposed to different environments where their sensors or actuators have induced errors. With the conclusions of this work, we set the initial point for future work on designing and developing methods to achieve robust reinforcement learning on the presence of real-world perturbances that might differ within a multi-robot system.


page 1

page 3


MultiRoboLearn: An open-source Framework for Multi-robot Deep Reinforcement Learning

It is well known that it is difficult to have a reliable and robust fram...

Ubiquitous Distributed Deep Reinforcement Learning at the Edge: Analyzing Byzantine Agents in Discrete Action Spaces

The integration of edge computing in next-generation mobile networks is ...

K-nearest Multi-agent Deep Reinforcement Learning for Collaborative Tasks with a Variable Number of Agents

Traditionally, the performance of multi-agent deep reinforcement learnin...

What Robot do I Need? Fast Co-Adaptation of Morphology and Control using Graph Neural Networks

The co-adaptation of robot morphology and behaviour becomes increasingly...

Learning to Fly – a Gym Environment with PyBullet Physics for Reinforcement Learning of Multi-agent Quadcopter Control

Robotic simulators are crucial for academic research and education as we...

Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey

Deep reinforcement learning has recently seen huge success across multip...

Learning Trajectories for Visual-Inertial System Calibration via Model-based Heuristic Deep Reinforcement Learning

Visual-inertial systems rely on precise calibrations of both camera intr...

Please sign up or login with your details

Forgot password? Click here to reset