Communication-Robust Multi-Agent Learning by Adaptable Auxiliary Multi-Agent Adversary Generation

by   Lei Yuan, et al.

Communication can promote coordination in cooperative Multi-Agent Reinforcement Learning (MARL). Nowadays, existing works mainly focus on improving the communication efficiency of agents, neglecting that real-world communication is much more challenging as there may exist noise or potential attackers. Thus the robustness of the communication-based policies becomes an emergent and severe issue that needs more exploration. In this paper, we posit that the ego system trained with auxiliary adversaries may handle this limitation and propose an adaptable method of Multi-Agent Auxiliary Adversaries Generation for robust Communication, dubbed MA3C, to obtain a robust communication-based policy. In specific, we introduce a novel message-attacking approach that models the learning of the auxiliary attacker as a cooperative problem under a shared goal to minimize the coordination ability of the ego system, with which every information channel may suffer from distinct message attacks. Furthermore, as naive adversarial training may impede the generalization ability of the ego system, we design an attacker population generation approach based on evolutionary learning. Finally, the ego system is paired with an attacker population and then alternatively trained against the continuously evolving attackers to improve its robustness, meaning that both the ego system and the attackers are adaptable. Extensive experiments on multiple benchmarks indicate that our proposed MA3C provides comparable or better robustness and generalization ability than other baselines.


page 4

page 9

page 11

page 13

page 14


Robust multi-agent coordination via evolutionary generation of auxiliary adversarial attackers

Cooperative multi-agent reinforcement learning (CMARL) has shown to be p...

Certifiably Robust Policy Learning against Adversarial Communication in Multi-agent Systems

Communication is important in many multi-agent reinforcement learning (M...

Robust Multi-agent Communication via Multi-view Message Certification

Many multi-agent scenarios require message sharing among agents to promo...

Gaussian Process Based Message Filtering for Robust Multi-Agent Cooperation in the Presence of Adversarial Communication

In this paper, we consider the problem of providing robustness to advers...

Widen The Backdoor To Let More Attackers In

As collaborative learning and the outsourcing of data collection become ...

Catch Me If You Can: Improving Adversaries in Cyber-Security With Q-Learning Algorithms

The ongoing rise in cyberattacks and the lack of skilled professionals i...

Multi-agent Continual Coordination via Progressive Task Contextualization

Cooperative Multi-agent Reinforcement Learning (MARL) has attracted sign...

Please sign up or login with your details

Forgot password? Click here to reset