Learning Social Learning

by   Kamal Ndousse, et al.

Social learning is a key component of human and animal intelligence. By taking cues from the behavior of experts in their environment, social learners can acquire sophisticated behavior and rapidly adapt to new circumstances. This paper investigates whether independent reinforcement learning (RL) agents in a multi-agent environment can use social learning to improve their performance using cues from other agents. We find that in most circumstances, vanilla model-free RL agents do not use social learning, even in environments in which individual exploration is expensive. We analyze the reasons for this deficiency, and show that by introducing a model-based auxiliary loss we are able to train agents to lever-age cues from experts to solve hard exploration tasks. The generalized social learning policy learned by these agents allows them to not only outperform the experts with which they trained, but also achieve better zero-shot transfer performance than solo learners when deployed to novel environments with experts. In contrast, agents that have not learned to rely on social learning generalize poorly and do not succeed in the transfer task. Further,we find that by mixing multi-agent and solo training, we can obtain agents that use social learning to out-perform agents trained alone, even when experts are not avail-able. This demonstrates that social learning has helped improve agents' representation of the task itself. Our results indicate that social learning can enable RL agents to not only improve performance on the task at hand, but improve generalization to novel environments.


page 2

page 3

page 4

page 5

page 7


Joint Attention for Multi-Agent Coordination and Social Learning

Joint attention - the ability to purposefully coordinate attention with ...

Collaborating with language models for embodied reasoning

Reasoning in a complex and ambiguous environment is a key goal for Reinf...

No Press Diplomacy: Modeling Multi-Agent Gameplay

Diplomacy is a seven-player non-stochastic, non-cooperative game, where ...

marl-jax: Multi-agent Reinforcement Leaning framework for Social Generalization

Recent advances in Reinforcement Learning (RL) have led to many exciting...

Theory-based Causal Transfer: Integrating Instance-level Induction and Abstract-level Structure Learning

Learning transferable knowledge across similar but different settings is...

Auxiliary Tasks and Exploration Enable ObjectNav

ObjectGoal Navigation (ObjectNav) is an embodied task wherein agents are...

Metacontrol for Adaptive Imagination-Based Optimization

Many machine learning systems are built to solve the hardest examples of...

Please sign up or login with your details

Forgot password? Click here to reset