Over-communicate no more: Situated RL agents learn concise communication protocols

by   Aleksandra Kalinowska, et al.

While it is known that communication facilitates cooperation in multi-agent settings, it is unclear how to design artificial agents that can learn to effectively and efficiently communicate with each other. Much research on communication emergence uses reinforcement learning (RL) and explores unsituated communication in one-step referential tasks – the tasks are not temporally interactive and lack time pressures typically present in natural communication. In these settings, agents may successfully learn to communicate, but they do not learn to exchange information concisely – they tend towards over-communication and an inefficient encoding. Here, we explore situated communication in a multi-step task, where the acting agent has to forgo an environmental action to communicate. Thus, we impose an opportunity cost on communication and mimic the real-world pressure of passing time. We compare communication emergence under this pressure against learning to communicate with a cost on articulation effort, implemented as a per-message penalty (fixed and progressively increasing). We find that while all tested pressures can disincentivise over-communication, situated communication does it most effectively and, unlike the cost on effort, does not negatively impact emergence. Implementing an opportunity cost on communication in a temporally extended environment is a step towards embodiment, and might be a pre-condition for incentivising efficient, human-like communication.


page 9

page 11

page 15

page 16


Learning to Communicate to Solve Riddles with Deep Distributed Recurrent Q-Networks

We propose deep distributed recurrent Q-networks (DDRQN), which enable t...

A Joint Learning and Communication Framework for Multi-Agent Reinforcement Learning over Noisy Channels

We propose a novel formulation of the "effectiveness problem" in communi...

Exploring Zero-Shot Emergent Communication in Embodied Multi-Agent Populations

Effective communication is an important skill for enabling information e...

Ease-of-Teaching and Language Structure from Emergent Communication

Artificial agents have been shown to learn to communicate when needed to...

Anti-efficient encoding in emergent communication

Despite renewed interest in emergent language simulations with neural ne...

Internal and external pressures on language emergence: least effort, object constancy and frequency

In previous work, artificial agents were shown to achieve almost perfect...

Context-dependent communication under environmental constraints

There is significant evidence that real-world communication cannot be re...

Please sign up or login with your details

Forgot password? Click here to reset