Concept Learning in Deep Reinforcement Learning

by   Diego Gomez, et al.

Deep reinforcement learning techniques have shown to be a promising path to solve very complex tasks that once were thought to be out of the realm of machines. However, while humans and animals learn incrementally during their lifetimes and exploit their experience to solve new tasks, standard deep learning methods specialize to solve only one task at a time and whatever information they acquire is hardly reusable in new situations. Given that any artificial agent would need such a generalization ability to deal with the complexities of the world, it is critical to understand what mechanisms give rise to this ability. We argue that one of the mechanisms humans rely on is the use of discrete conceptual representations to encode their sensory inputs. These representations group similar inputs in such a way that combined they provide a level of abstraction that is transverse to a wide variety of tasks, filtering out irrelevant information for their solution. Here, we show that it is possible to learn such concept-like representations by self-supervision, following an information-bottleneck approach, and that these representations accelerate the transference of skills by providing a prior that guides the policy optimization process. Our method is able to learn useful concepts in locomotive tasks that significantly reduce the number of optimization steps required, opening a new path to endow artificial agents with generalization abilities.


page 3

page 4

page 6

page 8

page 16

page 17

page 18

page 19


AIGenC: AI generalisation via creativity

This paper introduces a computational model of creative problem solving ...

Learning State Representations for Query Optimization with Deep Reinforcement Learning

Deep reinforcement learning is quickly changing the field of artificial ...

Language as an Abstraction for Hierarchical Deep Reinforcement Learning

Solving complex, temporally-extended tasks is a long-standing problem in...

Deep Reinforcement Learning for Dexterous Manipulation with Concept Networks

Deep reinforcement learning yields great results for a large array of pr...

Measuring and Characterizing Generalization in Deep Reinforcement Learning

Deep reinforcement-learning methods have achieved remarkable performance...

Perspective Taking in Deep Reinforcement Learning Agents

Perspective taking is the ability to take the point of view of another a...

Reward-Predictive Clustering

Recent advances in reinforcement-learning research have demonstrated imp...

Please sign up or login with your details

Forgot password? Click here to reset