HAMMER: Multi-Level Coordination of Reinforcement Learning Agents via Learned Messaging

by   Nikunj Gupta, et al.

Cooperative multi-agent reinforcement learning (MARL) has achieved significant results, most notably by leveraging the representation learning abilities of deep neural networks. However, large centralized approaches quickly become infeasible as the number of agents scale, and fully decentralized approaches can miss important opportunities for information sharing and coordination. Furthermore, not all agents are equal - in some cases, individual agents may not even have the ability to send communication to other agents or explicitly model other agents. This paper considers the case where there is a single, powerful, central agent that can observe the entire observation space, and there are multiple, low powered, local agents that can only receive local observations and cannot communicate with each other. The job of the central agent is to learn what message to send to different local agents, based on the global observations, not by centrally solving the entire problem and sending action commands, but by determining what additional information an individual agent should receive so that it can make a better decision. After explaining our MARL algorithm, hammer, and where it would be most applicable, we implement it in the cooperative navigation and multi-agent walker domains. Empirical results show that 1) learned communication does indeed improve system performance, 2) results generalize to multiple numbers of agents, and 3) results generalize to different reward structures.


Promoting Coordination Through Electing First-moveAgent in Multi-Agent Reinforcement Learning

Learning to coordinate among multiple agents is an essential problem in ...

Information-Bottleneck-Based Behavior Representation Learning for Multi-agent Reinforcement learning

In multi-agent deep reinforcement learning, extracting sufficient and co...

Multi-Agent Belief Sharing through Autonomous Hierarchical Multi-Level Clustering

Coordination in multi-agent systems is challenging for agile robots such...

Spatial Intention Maps for Multi-Agent Mobile Manipulation

The ability to communicate intention enables decentralized multi-agent r...

Learning to Improve Representations by Communicating About Perspectives

Effective latent representations need to capture abstract features of th...

Stigmergic Independent Reinforcement Learning for Multi-Agent Collaboration

With the rapid evolution of wireless mobile devices, it emerges stronger...

Multi-lingual agents through multi-headed neural networks

This paper considers cooperative Multi-Agent Reinforcement Learning, foc...

Please sign up or login with your details

Forgot password? Click here to reset