Complex Sequential Question Answering: Towards Learning to Converse Over Linked Question Answer Pairs with a Knowledge Graph

by   Amrita Saha, et al.

While conversing with chatbots, humans typically tend to ask many questions, a significant portion of which can be answered by referring to large-scale knowledge graphs (KG). While Question Answering (QA) and dialog systems have been studied independently, there is a need to study them closely to evaluate such real-world scenarios faced by bots involving both these tasks. Towards this end, we introduce the task of Complex Sequential QA which combines the two tasks of (i) answering factual questions through complex inferencing over a realistic-sized KG of millions of entities, and (ii) learning to converse through a series of coherently linked QA pairs. Through a labor intensive semi-automatic process, involving in-house and crowdsourced workers, we created a dataset containing around 200K dialogs with a total of 1.6M turns. Further, unlike existing large scale QA datasets which contain simple questions that can be answered from a single tuple, the questions in our dialogs require a larger subgraph of the KG. Specifically, our dataset has questions which require logical, quantitative, and comparative reasoning as well as their combinations. This calls for models which can: (i) parse complex natural language questions, (ii) use conversation context to resolve coreferences and ellipsis in utterances, (iii) ask for clarifications for ambiguous queries, and finally (iv) retrieve relevant subgraphs of the KG to answer such questions. However, our experiments with a combination of state of the art dialog and QA models show that they clearly do not achieve the above objectives and are inadequate for dealing with such complex real world settings. We believe that this new dataset coupled with the limitations of existing models as reported in this paper should encourage further research in Complex Sequential QA.


page 1

page 2

page 3

page 4


Context Transformer with Stacked Pointer Networks for Conversational Question Answering over Knowledge Graphs

Neural semantic parsing approaches have been widely used for Question An...

SPRING: Situated Conversation Agent Pretrained with Multimodal Questions from Incremental Layout Graph

Existing multimodal conversation agents have shown impressive abilities ...

Towards Collaborative Question Answering: A Preliminary Study

Knowledge and expertise in the real-world can be disjointedly owned. To ...

Variational Reasoning for Question Answering with Knowledge Graph

Knowledge graph (KG) is known to be helpful for the task of question ans...

ComQA: A Community-sourced Dataset for Complex Factoid Question Answering with Paraphrase Clusters

To bridge the gap between the capabilities of the state-of-the-art in fa...

KQA Pro: A Large Diagnostic Dataset for Complex Question Answering over Knowledge Base

Complex question answering over knowledge base (Complex KBQA) is challen...

QUADRo: Dataset and Models for QUestion-Answer Database Retrieval

An effective paradigm for building Automated Question Answering systems ...

Please sign up or login with your details

Forgot password? Click here to reset