Backdoor Attacks on Multiagent Collaborative Systems

by   Shuo Chen, et al.

Backdoor attacks on reinforcement learning implant a backdoor in a victim agent's policy. Once the victim observes the trigger signal, it will switch to the abnormal mode and fail its task. Most of the attacks assume the adversary can arbitrarily modify the victim's observations, which may not be practical. One work proposes to let one adversary agent use its actions to affect its opponent in two-agent competitive games, so that the opponent quickly fails after observing certain trigger actions. However, in multiagent collaborative systems, agents may not always be able to observe others. When and how much the adversary agent can affect others are uncertain, and we want the adversary agent to trigger others for as few times as possible. To solve this problem, we first design a novel training framework to produce auxiliary rewards that measure the extent to which the other agents'observations being affected. Then we use the auxiliary rewards to train a trigger policy which enables the adversary agent to efficiently affect the others' observations. Given these affected observations, we further train the other agents to perform abnormally. Extensive experiments demonstrate that the proposed method enables the adversary agent to lure the others into the abnormal mode with only a few actions.


page 7

page 9

page 10

page 11


BACKDOORL: Backdoor Attack against Competitive Reinforcement Learning

Recent research has confirmed the feasibility of backdoor attacks in dee...

Understanding Adversarial Attacks on Observations in Deep Reinforcement Learning

Recent works demonstrate that deep reinforcement learning (DRL) models a...

Adversary agent reinforcement learning for pursuit-evasion

A reinforcement learning environment with adversary agents is proposed i...

Learning an Adversary's Actions for Secret Communication

Secure communication over a wiretap channel is investigated, in which an...

Virtuously Safe Reinforcement Learning

We show that when a third party, the adversary, steps into the two-party...

Moody Learners – Explaining Competitive Behaviour of Reinforcement Learning Agents

Designing the decision-making processes of artificial agents that are in...

Collaborative Decision Making Using Action Suggestions

The level of autonomy is increasing in systems spanning multiple domains...

Please sign up or login with your details

Forgot password? Click here to reset