Can Large Language Models Infer Causation from Correlation?

by   Zhijing Jin, et al.

Causal inference is one of the hallmarks of human intelligence. While the field of CausalNLP has attracted much interest in the recent years, existing causal inference datasets in NLP primarily rely on discovering causality from empirical knowledge (e.g., commonsense knowledge). In this work, we propose the first benchmark dataset to test the pure causal inference skills of large language models (LLMs). Specifically, we formulate a novel task Corr2Cause, which takes a set of correlational statements and determines the causal relationship between the variables. We curate a large-scale dataset of more than 400K samples, on which we evaluate seventeen existing LLMs. Through our experiments, we identify a key shortcoming of LLMs in terms of their causal inference skills, and show that these models achieve almost close to random performance on the task. This shortcoming is somewhat mitigated when we try to re-purpose LLMs for this skill via finetuning, but we find that these models still fail to generalize – they can only perform causal inference in in-distribution settings when variable names and textual expressions used in the queries are similar to those in the training set, but fail in out-of-distribution settings generated by perturbing these queries. Corr2Cause is a challenging task for LLMs, and would be helpful in guiding future research on improving LLMs' pure reasoning skills and generalizability. Our data is at Our code is at


page 1

page 2

page 3

page 4


Causal Parrots: Large Language Models May Talk Causality But Are Not Causal

Some argue scale is all what is needed to achieve AI, covering even caus...

Causal Inference for Banking Finance and Insurance A Survey

Causal Inference plays an significant role in explaining the decisions t...

Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond

A fundamental goal of scientific research is to learn about causal relat...

Bayesian Causal Inference

We address the problem of two-variable causal inference. This task is to...

Everything Has a Cause: Leveraging Causal Inference in Legal Text Analysis

Causal inference is the process of capturing cause-effect relationship a...

FAIR: A Causal Framework for Accurately Inferring Judgments Reversals

Artificial intelligence researchers have made significant advances in le...

Veritas: Answering Causal Queries from Video Streaming Traces

In this paper, we seek to answer what-if questions - i.e., given recorde...

Please sign up or login with your details

Forgot password? Click here to reset