Incremental Improvement of a Question Answering System by Re-ranking Answer Candidates using Machine Learning

by   Michael Barz, et al.

We implement a method for re-ranking top-10 results of a state-of-the-art question answering (QA) system. The goal of our re-ranking approach is to improve the answer selection given the user question and the top-10 candidates. We focus on improving deployed QA systems that do not allow re-training or re-training comes at a high cost. Our re-ranking approach learns a similarity function using n-gram based features using the query, the answer and the initial system confidence as input. Our contributions are: (1) we generate a QA training corpus starting from 877 answers from the customer care domain of T-Mobile Austria, (2) we implement a state-of-the-art QA pipeline using neural sentence embeddings that encode queries in the same space than the answer index, and (3) we evaluate the QA pipeline and our re-ranking approach using a separately provided test set. The test set can be considered to be available after deployment of the system, e.g., based on feedback of users. Our results show that the system performance, in terms of top-n accuracy and the mean reciprocal rank, benefits from re-ranking using gradient boosted regression trees. On average, the mean reciprocal rank improves by 9.15


Using Interactive Feedback to Improve the Accuracy and Explainability of Question Answering Systems Post-Deployment

Most research on question answering focuses on the pre-deployment stage;...

Evidence Aggregation for Answer Re-Ranking in Open-Domain Question Answering

A popular recent approach to answering open-domain questions is to first...

RankQA: Neural Question Answering with Answer Re-Ranking

The conventional paradigm in neural question answering (QA) for narrativ...

A Case Based Reasoning Approach for Answer Reranking in Question Answering

In this document I present an approach to answer validation and rerankin...

Double Retrieval and Ranking for Accurate Question Answering

Recent work has shown that an answer verification step introduced in Tra...

Hierarchical Ranking for Answer Selection

Answer selection is a task to choose the positive answers from a pool of...

PReGAN: Answer Oriented Passage Ranking with Weakly Supervised GAN

Beyond topical relevance, passage ranking for open-domain factoid questi...

Please sign up or login with your details

Forgot password? Click here to reset