Visual Referential Games Further the Emergence of Disentangled Representations

by   Kevin Denamganai, et al.

Natural languages are powerful tools wielded by human beings to communicate information. Among their desirable properties, compositionality has been the main focus in the context of referential games and variants, as it promises to enable greater systematicity to the agents which would wield it. The concept of disentanglement has been shown to be of paramount importance to learned representations that generalise well in deep learning, and is thought to be a necessary condition to enable systematicity. Thus, this paper investigates how do compositionality at the level of the emerging languages, disentanglement at the level of the learned representations, and systematicity relate to each other in the context of visual referential games. Firstly, we find that visual referential games that are based on the Obverter architecture outperforms state-of-the-art unsupervised learning approach in terms of many major disentanglement metrics. Secondly, we expand the previously proposed Positional Disentanglement (PosDis) metric for compositionality to (re-)incorporate some concerns pertaining to informativeness and completeness features found in the Mutual Information Gap (MIG) disentanglement metric it stems from. This extension allows for further discrimination between the different kind of compositional languages that emerge in the context of Obverter-based referential games, in a way that neither the referential game accuracy nor previous metrics were able to capture. Finally we investigate whether the resulting (emergent) systematicity, as measured by zero-shot compositional learning tests, correlates with any of the disentanglement and compositionality metrics proposed so far. Throughout the training process, statically significant correlation coefficients can be found both positive and negative depending on the moment of the measure.


page 16

page 17

page 18


On (Emergent) Systematic Generalisation and Compositionality in Visual Referential Games with Straight-Through Gumbel-Softmax Estimator

The drivers of compositionality in artificial languages that emerge when...

Internal and external pressures on language emergence: least effort, object constancy and frequency

In previous work, artificial agents were shown to achieve almost perfect...

ReferentialGym: A Nomenclature and Framework for Language Emergence Grounding in (Visual) Referential Games

Natural languages are powerful tools wielded by human beings to communic...

Emergent Communication of Generalizations

To build agents that can collaborate effectively with others, recent res...

Compositional Obverter Communication Learning From Raw Visual Input

One of the distinguishing aspects of human language is its compositional...

Emergent Communication: Generalization and Overfitting in Lewis Games

Lewis signaling games are a class of simple communication games for simu...

Please sign up or login with your details

Forgot password? Click here to reset