SCAN: Learning Abstract Hierarchical Compositional Visual Concepts

by   Irina Higgins, et al.

The natural world is infinitely diverse, yet this diversity arises from a relatively small set of coherent properties and rules, such as the laws of physics or chemistry. We conjecture that biological intelligent systems are able to survive within their diverse environments by discovering the regularities that arise from these rules primarily through unsupervised experiences, and representing this knowledge as abstract concepts. Such representations possess useful properties of compositionality and hierarchical organisation, which allow intelligent agents to recombine a finite set of conceptual building blocks into an exponentially large set of useful new concepts. This paper describes SCAN (Symbol-Concept Association Network), a new framework for learning such concepts in the visual domain. We first use the previously published beta-VAE (Higgins et al., 2017a) architecture to learn a disentangled representation of the latent structure of the visual world, before training SCAN to extract abstract concepts grounded in such disentangled visual primitives through fast symbol association. Our approach requires very few pairings between symbols and images and makes no assumptions about the choice of symbol representations. Once trained, SCAN is capable of multimodal bi-directional inference, generating a diverse set of image samples from symbolic descriptions and vice versa. It also allows for traversal and manipulation of the implicit hierarchy of compositional visual concepts through symbolic instructions and learnt logical recombination operations. Such manipulations enable SCAN to invent and learn novel visual concepts through recombination of the few learnt concepts.


page 3

page 11

page 12

page 13

page 17

page 18

page 19

page 21


Learning Task-General Representations with Generative Neuro-Symbolic Modeling

A hallmark of human intelligence is the ability to interact directly wit...

Discovering the Compositional Structure of Vector Representations with Role Learning Networks

Neural networks (NNs) are able to perform tasks that rely on composition...

Neural Algebra of Classifiers

The world is fundamentally compositional, so it is natural to think of v...

Generation of concept-representative symbols

The visual representation of concepts or ideas through the use of simple...

The Roles of Symbols in Neural-based AI: They are Not What You Think!

We propose that symbols are first and foremost external communication to...

Compositional Law Parsing with Latent Random Functions

Human cognition has compositionality. We understand a scene by decomposi...

On the Learnability of Physical Concepts: Can a Neural Network Understand What's Real?

We revisit the classic signal-to-symbol barrier in light of the remarkab...

Code Repositories

Please sign up or login with your details

Forgot password? Click here to reset