Centering the Margins: Outlier-Based Identification of Harmed Populations in Toxicity Detection

by   Vyoma Raman, et al.

A standard method for measuring the impacts of AI on marginalized communities is to determine performance discrepancies between specified demographic groups. These approaches aim to address harms toward vulnerable groups, but they obscure harm patterns faced by intersectional subgroups or shared across demographic groups. We instead operationalize "the margins" as data points that are statistical outliers due to having demographic attributes distant from the "norm" and measure harms toward these outliers. We propose a Group-Based Performance Disparity Index (GPDI) that measures the extent to which a subdivision of a dataset into subgroups identifies those facing increased harms. We apply our approach to detecting disparities in toxicity detection and find that text targeting outliers is 28 toxicity examined. We also discover that model performance is consistently worse for demographic outliers, with disparities in error between outliers and non-outliers ranging from 28 analysis has comparable or higher GPDI than traditional subgroup-based analyses, suggesting that outlier analysis enhances identification of subgroups facing greater harms. Finally, we find that minoritized racial and religious groups are most associated with outliers, which suggests that outlier analysis is particularly beneficial for identifying harms against those groups.


page 8

page 9


Outlier-Robust Group Inference via Gradient Space Clustering

Traditional machine learning models focus on achieving good performance ...

Same Same, But Different: Conditional Multi-Task Learning for Demographic-Specific Toxicity Detection

Algorithmic bias often arises as a result of differential subgroup valid...

Wiki-based Communities of Interest: Demographics and Outliers

In this paper, we release data about demographic information and outlier...

Outlier Detection Using a Novel method: Quantum Clustering

We propose a new assumption in outlier detection: Normal data instances ...

Interpreting Outliers: Localized Logistic Regression for Density Ratio Estimation

We propose an inlier-based outlier detection method capable of both iden...

Reply to Comment on "TVOR: Finding Discrete Total Variation Outliers among Histograms"

In this paper, we respond to a critique of one of our papers previously ...

Application of Markov Structure of Genomes to Outlier Identification and Read Classification

In this paper we apply the structure of genomes as second-order Markov p...

Please sign up or login with your details

Forgot password? Click here to reset