Heterogeneous Embodied Multi-Agent Collaboration

by   Xinzhu Liu, et al.

Multi-agent embodied tasks have recently been studied in complex indoor visual environments. Collaboration among multiple agents can improve work efficiency and has significant practical value. However, most of the existing research focuses on homogeneous multi-agent tasks. Compared with homogeneous agents, heterogeneous agents can leverage their different capabilities to allocate corresponding sub-tasks and cooperate to complete complex tasks. Heterogeneous multi-agent tasks are common in real-world scenarios, and the collaboration strategy among heterogeneous agents is a challenging and important problem to be solved. To study collaboration among heterogeneous agents, we propose the heterogeneous multi-agent tidying-up task, in which multiple heterogeneous agents with different capabilities collaborate with each other to detect misplaced objects and place them in reasonable locations. This is a demanding task since it requires agents to make the best use of their different capabilities to conduct reasonable task planning and complete the whole task. To solve this task, we build a heterogeneous multi-agent tidying-up benchmark dataset in a large number of houses with multiple rooms based on ProcTHOR-10K. We propose the hierarchical decision model based on misplaced object detection, reasonable receptacle prediction, as well as the handshake-based group communication mechanism. Extensive experiments are conducted to demonstrate the effectiveness of the proposed model. The project's website and videos of experiments can be found at https://hetercol.github.io/.


page 1

page 4

page 5

page 10

page 11


Multi-Agent Embodied Visual Semantic Navigation with Scene Prior Knowledge

In visual semantic navigation, the robot navigates to a target object wi...

MindAgent: Emergent Gaming Interaction

Large Language Models (LLMs) have the capacity of performing complex sch...

Collaborative Visual Navigation

As a fundamental problem for Artificial Intelligence, multi-agent system...

Two Body Problem: Collaborative Visual Task Completion

Collaboration is a necessary skill to perform tasks that are beyond one ...

Verse: A Python library for reasoning about multi-agent hybrid system scenarios

We present the Verse library with the aim of making hybrid system verifi...

Autonomous Task Planning for Heterogeneous Multi-Agent Systems

This paper presents a solution to the automatic task planning problem fo...

Gamified and Self-Adaptive Applications for the Common Good: Research Challenges Ahead

Motivational digital systems offer capabilities to engage and motivate e...

Please sign up or login with your details

Forgot password? Click here to reset