Learning Causal Mechanisms through Orthogonal Neural Networks

A fundamental feature of human intelligence is the ability to infer high-level abstractions from low-level sensory data. An essential component of such inference is the ability to discover modularized generative mechanisms. Despite many efforts to use statistical learning and pattern recognition for finding disentangled factors, arguably human intelligence remains unmatched in this area. In this paper, we investigate a problem of learning, in a fully unsupervised manner, the inverse of a set of independent mechanisms from distorted data points. We postulate, and justify this claim with experimental results, that an important weakness of existing machine learning solutions lies in the insufficiency of cross-module diversification. Addressing this crucial discrepancy between human and machine intelligence is an important challenge for pattern recognition systems. To this end, our work proposes an unsupervised method that discovers and disentangles a set of independent mechanisms from unlabeled data, and learns how to invert them. A number of experts compete against each other for individual data points in an adversarial setting: one that best inverses the (unknown) generative mechanism is the winner. We demonstrate that introducing an orthogonalization layer into the expert architectures enforces additional diversity in the outputs, leading to significantly better separability. Moreover, we propose a procedure for relocating data points between experts to further prevent any one from claiming multiple mechanisms. We experimentally illustrate that these techniques allow discovery and modularization of much less pronounced transformations, in addition to considerably faster convergence.


page 6

page 11


Learning Independent Causal Mechanisms

Independent causal mechanisms are a central concept in the study of caus...

Augmenting C. elegans Microscopic Dataset for Accelerated Pattern Recognition

The detection of cell shape changes in 3D time-lapse images of complex t...

IQ of Neural Networks

IQ tests are an accepted method for assessing human intelligence. The te...

Inferencing Based on Unsupervised Learning of Disentangled Representations

Combining Generative Adversarial Networks (GANs) with encoders that lear...

KANDINSKYPatterns – An experimental exploration environment for Pattern Analysis and Machine Intelligence

Machine intelligence is very successful at standard recognition tasks wh...

Boundary Optimizing Network (BON)

Despite all the success that deep neural networks have seen in classifyi...

Ab initio Algorithmic Causal Deconvolution of Intertwined Programs and Networks by Generative Mechanism

To extract and learn representations leading to generative mechanisms fr...

Please sign up or login with your details

Forgot password? Click here to reset