Generalized Label Shift Correction via Minimum Uncertainty Principle: Theory and Algorithm

by   You-Wei Luo, et al.

As a fundamental problem in machine learning, dataset shift induces a paradigm to learn and transfer knowledge under changing environment. Previous methods assume the changes are induced by covariate, which is less practical for complex real-world data. We consider the Generalized Label Shift (GLS), which provides an interpretable insight into the learning and transfer of desirable knowledge. Current GLS methods: 1) are not well-connected with the statistical learning theory; 2) usually assume the shifting conditional distributions will be matched with an implicit transformation, but its explicit modeling is unexplored. In this paper, we propose a conditional adaptation framework to deal with these challenges. From the perspective of learning theory, we prove that the generalization error of conditional adaptation is lower than previous covariate adaptation. Following the theoretical results, we propose the minimum uncertainty principle to learn conditional invariant transformation via discrepancy optimization. Specifically, we propose the conditional metric operator on Hilbert space to characterize the distinctness of conditional distributions. For finite observations, we prove that the empirical estimation is always well-defined and will converge to underlying truth as sample size increases. The results of extensive experiments demonstrate that the proposed model achieves competitive performance under different GLS scenarios.


page 1

page 6

page 11

page 13


Conditional Bures Metric for Domain Adaptation

As a vital problem in classification-oriented transfer, unsupervised dom...

Generalized Domain Adaptation with Covariate and Label Shift CO-ALignment

Unsupervised knowledge transfer has a great potential to improve the gen...

Mitigating Both Covariate and Conditional Shift for Domain Generalization

Domain generalization (DG) aims to learn a model on several source domai...

On the Use of Sparse Filtering for Covariate Shift Adaptation

In this paper we formally analyse the use of sparse filtering algorithms...

Towards Backwards-Compatible Data with Confounded Domain Adaptation

Most current domain adaptation methods address either covariate shift or...

Stable Learning via Sparse Variable Independence

The problem of covariate-shift generalization has attracted intensive re...

DIRL: Domain-Invariant Representation Learning for Sim-to-Real Transfer

Generating large-scale synthetic data in simulation is a feasible altern...

Please sign up or login with your details

Forgot password? Click here to reset