Learning Invariant Representations and Risks for Semi-supervised Domain Adaptation

by   Bo Li, et al.

The success of supervised learning hinges on the assumption that the training and test data come from the same underlying distribution, which is often not valid in practice due to potential distribution shift. In light of this, most existing methods for unsupervised domain adaptation focus on achieving domain-invariant representations and small source domain error. However, recent works have shown that this is not sufficient to guarantee good generalization on the target domain, and in fact, is provably detrimental under label distribution shift. Furthermore, in many real-world applications it is often feasible to obtain a small amount of labeled data from the target domain and use them to facilitate model training with source data. Inspired by the above observations, in this paper we propose the first method that aims to simultaneously learn invariant representations and risks under the setting of semi-supervised domain adaptation (Semi-DA). First, we provide a finite sample bound for both classification and regression problems under Semi-DA. The bound suggests a principled way to obtain target generalization, i.e. by aligning both the marginal and conditional distributions across domains in feature space. Motivated by this, we then introduce the LIRR algorithm for jointly Learning Invariant Representations and Risks. Finally, extensive experiments are conducted on both classification and regression tasks, which demonstrates LIRR consistently achieves state-of-the-art performance and significant improvements compared with the methods that only learn invariant representations or invariant risks.


page 9

page 22


On Learning Invariant Representation for Domain Adaptation

Due to the ability of deep neural nets to learn rich representations, re...

Hidden Covariate Shift: A Minimal Assumption For Domain Adaptation

Unsupervised Domain Adaptation aims to learn a model on a source domain ...

On Learning Domain-Invariant Representations for Transfer Learning with Multiple Sources

Domain adaptation (DA) benefits from the rigorous theoretical works that...

Semi-Generative Modelling: Domain Adaptation with Cause and Effect Features

This paper presents a novel, causally-inspired approach to domain adapta...

Learning List-Level Domain-Invariant Representations for Ranking

Domain adaptation aims to transfer the knowledge acquired by models trai...

Prominent Roles of Conditionally Invariant Components in Domain Adaptation: Theory and Algorithms

Domain adaptation (DA) is a statistical learning problem that arises whe...

Algorithms and Theory for Supervised Gradual Domain Adaptation

The phenomenon of data distribution evolving over time has been observed...

Please sign up or login with your details

Forgot password? Click here to reset