MEMO: A Deep Network for Flexible Combination of Episodic Memories

by   Andrea Banino, et al.

Recent research developing neural network architectures with external memory have often used the benchmark bAbI question and answering dataset which provides a challenging number of tasks requiring reasoning. Here we employed a classic associative inference task from the memory-based reasoning neuroscience literature in order to more carefully probe the reasoning capacity of existing memory-augmented architectures. This task is thought to capture the essence of reasoning – the appreciation of distant relationships among elements distributed across multiple facts or memories. Surprisingly, we found that current architectures struggle to reason over long distance associations. Similar results were obtained on a more complex task involving finding the shortest path between nodes in a path. We therefore developed MEMO, an architecture endowed with the capacity to reason over longer distances. This was accomplished with the addition of two novel components. First, it introduces a separation between memories (facts) stored in external memory and the items that comprise these facts in external memory. Second, it makes use of an adaptive retrieval mechanism, allowing a variable number of "memory hops" before the answer is produced. MEMO is capable of solving our novel reasoning tasks, as well as match state of the art results in bAbI.


page 1

page 2

page 3

page 4


Towards Neural Network-based Reasoning

We propose Neural Reasoner, a framework for neural network-based reasoni...

Finding ReMO (Related Memory Object): A Simple Neural Architecture for Text based Reasoning

To solve the text-based question and answering task that requires relati...

Incorporating External Knowledge to Answer Open-Domain Visual Questions with Dynamic Memory Networks

Visual Question Answering (VQA) has attracted much attention since it of...

Out of the Box: Reasoning with Graph Convolution Nets for Factual Visual Question Answering

Accurately answering a question about a given image requires combining o...

Seeing is Knowing! Fact-based Visual Question Answering using Knowledge Graph Embeddings

Fact-based Visual Question Answering (FVQA), a challenging variant of VQ...

Query-Reduction Networks for Question Answering

In this paper, we study the problem of question answering when reasoning...

Self-Assttentive Associative Memory

Heretofore, neural networks with external memory are restricted to singl...

Please sign up or login with your details

Forgot password? Click here to reset