BERT-kNN: Adding a kNN Search Component to Pretrained Language Models for Better QA

05/02/2020
by   Nora Kassner, et al.
0

Khandelwal et al. (2020) show that a k-nearest-neighbor (kNN) component improves language modeling performance. We use this idea for open domain question answering (QA). To improve the recall of facts stated in the training text, we combine BERT (Devlin et al., 2019) with a kNN search over a large corpus. Our contributions are as follows. i) We outperform BERT on cloze-style QA by large margins without any further training. ii) We show that BERT often identifies the correct response category (e.g., central European city), but only kNN recovers the factually correct answer (e.g., "Vienna").

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset