The Argument Reasoning Comprehension Task
share
Reasoning is a crucial part of natural language argumentation. In order to comprehend an argument, one has to reconstruct and analyze its reasoning. As arguments are highly contextualized, most reasoning-related content is left implicit and usually presupposed. Thus, argument comprehension requires not only language understanding and logic skills, but it also heavily depends on common sense. In this article we define a new task, argument reasoning comprehension. Given a natural language argument with a reason and a claim, the goal is to choose the correct implicit reasoning from two options. The challenging factor is that both options are plausible and lexically very close while leading to contradicting claims. To provide an empirical common ground for the task, we propose a complex, yet scalable crowdsourcing process, and we create a new freely licensed dataset based on authentic arguments from news comments. While the resulting 2k high-quality instances are also suitable for other argumentation-related tasks, such as stance detection, argument component identification, and abstractive argument summarization, we focus on the argument reasoning comprehension task and experiment with several systems based on neural attention and language models. Our results clearly reveal that current methods lack the capability to solve the task.
READ FULL TEXT