Deconstructing Claims of Post-Treatment Bias in Observational Studies of Discrimination

by   Johann Gaebler, et al.

In studies of discrimination, researchers often seek to estimate a causal effect of race or gender on outcomes. For example, in the criminal justice context, one might ask whether arrested individuals would have been subsequently charged or convicted had they been a different race. It has long been known that such counterfactual questions face measurement challenges related to omitted-variable bias, and conceptual challenges related to the definition of causal estimands for largely immutable characteristics. Another concern, raised most recently in Knox et al. [2020], is post-treatment bias. The authors argue that many studies of discrimination condition on intermediate outcomes, like being arrested, which themselves may be the product of discrimination, corrupting statistical estimates. Here we show that the Knox et al. critique is itself flawed, suffering from a mathematical error. Within their causal framework, we prove that a primary quantity of interest in discrimination studies is nonparametrically identifiable under a standard ignorability condition that is common in causal inference with observational data. More generally, though, we argue that it is often problematic to conceptualize discrimination in terms of a causal effect of protected attributes on decisions. We present an alternative perspective that avoids the common statistical difficulties, and which closely relates to long-standing legal and economic theories of disparate impact. We illustrate these ideas both with synthetic data and by analyzing the charging decisions of a prosecutor's office in a large city in the United States.


A note on post-treatment selection in studying racial discrimination in policing

We discuss some causal estimands used to study racial discrimination in ...

Omitted and Included Variable Bias in Tests for Disparate Impact

Policymakers often seek to gauge discrimination against groups defined b...

Promises and Challenges of Causality for Ethical Machine Learning

In recent years, there has been increasing interest in causal reasoning ...

A Causal Linear Model to Quantify Edge Unfairness for Unfair Edge Prioritization and Discrimination Removal

The dataset can be generated by an unfair mechanism in numerous settings...

Fairness in Algorithmic Decision Making: An Excursion Through the Lens of Causality

As virtually all aspects of our lives are increasingly impacted by algor...

Treatment effect bias from sample snooping: blinding outcomes is neither necessary nor sufficient

Popular guidance on observational data analysis states that outcomes sho...

Unwarranted Gender Disparity in Online P2P Lending: Evidence of Affirmative Action

Closing the gender gap in financial access is important. Most research t...

Please sign up or login with your details

Forgot password? Click here to reset