The Shapley Value of Classifiers in Ensemble Games

by   Benedek Rozemberczki, et al.

How do we decide the fair value of individual classifiers in an ensemble model? We introduce a new class of transferable utility cooperative games to answer this question. The players in ensemble games are pre-trained binary classifiers that collaborate in an ensemble to correctly label points from a dataset. We design Troupe a scalable algorithm that designates payoffs to individual models based on the Shapley value of those in the ensemble game. We show that the approximate Shapley value of classifiers in these games is an adequate measure for selecting a subgroup of highly predictive models. In addition, we introduce the Shapley entropy a new metric to quantify the heterogeneity of machine learning ensembles when it comes to model quality. We analytically prove that our Shapley value approximation algorithm is accurate and scales to large ensembles and big data. Experimental results on graph classification tasks establish that Troupe gives precise estimates of the Shapley value in ensemble games. We demonstrate that the Shapley value can be used for pruning large ensembles, show that complex classifiers have a prime role in correct and incorrect classification decisions, and provide evidence that adversarial models receive a low valuation.


page 1

page 2

page 3

page 4


Smart Data based Ensemble for Imbalanced Big Data Classification

Big Data scenarios pose a new challenge to traditional data mining algor...

Ensemble Pruning via Margin Maximization

Ensemble models refer to methods that combine a typically large number o...

Pathologies of Predictive Diversity in Deep Ensembles

Classical results establish that ensembles of small models benefit when ...

Leveraging Linear Independence of Component Classifiers: Optimizing Size and Prediction Accuracy for Online Ensembles

Ensembles, which employ a set of classifiers to enhance classification a...

Estimating Confidence of Predictions of Individual Classifiers and Their Ensembles for the Genre Classification Task

Genre identification is a subclass of non-topical text classification. T...

Noise-Resilient Ensemble Learning using Evidence Accumulation Clustering

Ensemble Learning methods combine multiple algorithms performing the sam...

Please sign up or login with your details

Forgot password? Click here to reset