MADiff: Offline Multi-agent Learning with Diffusion Models

05/27/2023
by   Zhengbang Zhu, et al.
0

Diffusion model (DM), as a powerful generative model, recently achieved huge success in various scenarios including offline reinforcement learning, where the policy learns to conduct planning by generating trajectory in the online evaluation. However, despite the effectiveness shown for single-agent learning, it remains unclear how DMs can operate in multi-agent problems, where agents can hardly complete teamwork without good coordination by independently modeling each agent's trajectories. In this paper, we propose MADiff, a novel generative multi-agent learning framework to tackle this problem. MADiff is realized with an attention-based diffusion model to model the complex coordination among behaviors of multiple diffusion agents. To the best of our knowledge, MADiff is the first diffusion-based multi-agent offline RL framework, which behaves as both a decentralized policy and a centralized controller, which includes opponent modeling and can be used for multi-agent trajectory prediction. MADiff takes advantage of the powerful generative ability of diffusion while well-suited in modeling complex multi-agent interactions. Our experiments show the superior performance of MADiff compared to baseline algorithms in a range of multi-agent learning tasks.

READ FULL TEXT
research
07/21/2023

Offline Multi-Agent Reinforcement Learning with Implicit Global-to-Local Value Regularization

Offline reinforcement learning (RL) has received considerable attention ...
research
03/20/2018

Generative Multi-Agent Behavioral Cloning

We propose and study the problem of generative multi-agent behavioral cl...
research
07/04/2023

Beyond Conservatism: Diffusion Policies in Offline Multi-agent Reinforcement Learning

We present a novel Diffusion Offline Multi-agent Model (DOM2) for offlin...
research
07/12/2023

Diffusion Based Multi-Agent Adversarial Tracking

Target tracking plays a crucial role in real-world scenarios, particular...
research
03/22/2023

EDGI: Equivariant Diffusion for Planning with Embodied Agents

Embodied agents operate in a structured world, often solving tasks with ...
research
01/02/2013

MANCaLog: A Logic for Multi-Attribute Network Cascades (Technical Report)

The modeling of cascade processes in multi-agent systems in the form of ...
research
05/26/2023

A Model-Based Solution to the Offline Multi-Agent Reinforcement Learning Coordination Problem

Training multiple agents to coordinate is an important problem with appl...

Please sign up or login with your details

Forgot password? Click here to reset