Bipartite Ranking Fairness through a Model Agnostic Ordering Adjustment

by   Sen Cui, et al.

Algorithmic fairness has been a serious concern and received lots of interest in machine learning community. In this paper, we focus on the bipartite ranking scenario, where the instances come from either the positive or negative class and the goal is to learn a ranking function that ranks positive instances higher than negative ones. While there could be a trade-off between fairness and performance, we propose a model agnostic post-processing framework xOrder for achieving fairness in bipartite ranking and maintaining the algorithm classification performance. In particular, we optimize a weighted sum of the utility as identifying an optimal warping path across different protected groups and solve it through a dynamic programming process. xOrder is compatible with various classification models and ranking fairness metrics, including supervised and unsupervised fairness metrics. In addition to binary groups, xOrder can be applied to multiple protected groups. We evaluate our proposed algorithm on four benchmark data sets and two real-world patient electronic health record repositories. xOrder consistently achieves a better balance between the algorithm utility and ranking fairness on a variety of datasets with different metrics. From the visualization of the calibrated ranking scores, xOrder mitigates the score distribution shifts of different groups compared with baselines. Moreover, additional analytical results verify that xOrder achieves a robust performance when faced with fewer samples and a bigger difference between training and testing ranking score distributions.


xOrder: A Model Agnostic Post-Processing Framework for Achieving Ranking Fairness While Maintaining Algorithm Utility

Algorithmic fairness has received lots of interests in machine learning ...

From Soft Classifiers to Hard Decisions: How fair can we be?

A popular methodology for building binary decision-making classifiers in...

The Fairness of Risk Scores Beyond Classification: Bipartite Ranking and the xAUC Metric

Where machine-learned predictive risk scores inform high-stakes decision...

Fair Ranking with Noisy Protected Attributes

The fair-ranking problem, which asks to rank a given set of items to max...

Achieving Downstream Fairness with Geometric Repair

Consider a scenario where some upstream model developer must train a fai...

Top Rank Optimization in Linear Time

Bipartite ranking aims to learn a real-valued ranking function that orde...

Societal Biases in Retrieved Contents: Measurement Framework and Adversarial Mitigation for BERT Rankers

Societal biases resonate in the retrieved contents of information retrie...

Please sign up or login with your details

Forgot password? Click here to reset