Deep Exemplar Networks for VQA and VQG

by   Badri N. Patro, et al.

In this paper, we consider the problem of solving semantic tasks such as `Visual Question Answering' (VQA), where one aims to answers related to an image and `Visual Question Generation' (VQG), where one aims to generate a natural question pertaining to an image. Solutions for VQA and VQG tasks have been proposed using variants of encoder-decoder deep learning based frameworks that have shown impressive performance. Humans however often show generalization by relying on exemplar based approaches. For instance, the work by Tversky and Kahneman suggests that humans use exemplars when making categorizations and decisions. In this work, we propose the incorporation of exemplar based approaches towards solving these problems. Specifically, we incorporate exemplar based approaches and show that an exemplar based module can be incorporated in almost any of the deep learning architectures proposed in the literature and the addition of such a block results in improved performance for solving these tasks. Thus, just as the incorporation of attention is now considered de facto useful for solving these tasks, similarly, incorporating exemplars also can be considered to improve any proposed architecture for solving this task. We provide extensive empirical analysis for the same through various architectures, ablations, and state of the art comparisons.


page 9

page 14

page 16

page 17

page 18

page 19

page 21

page 22


An Improved Attention for Visual Question Answering

We consider the problem of Visual Question Answering (VQA). Given an ima...

Survey of Visual Question Answering: Datasets and Techniques

Visual question answering (or VQA) is a new and exciting problem that co...

Latent Variable Models for Visual Question Answering

Conventional models for Visual Question Answering (VQA) explore determin...

Analyzing the Behavior of Visual Question Answering Models

Recently, a number of deep-learning based models have been proposed for ...

On the Significance of Question Encoder Sequence Model in the Out-of-Distribution Performance in Visual Question Answering

Generalizing beyond the experiences has a significant role in developing...

Towards Efficient and Elastic Visual Question Answering with Doubly Slimmable Transformer

Transformer-based approaches have shown great success in visual question...

An Empirical Evaluation of various Deep Learning Architectures for Bi-Sequence Classification Tasks

Several tasks in argumentation mining and debating, question-answering, ...

Please sign up or login with your details

Forgot password? Click here to reset