Pareto Domain Adaptation

12/08/2021
by   Fangrui Lv, et al.
7

Domain adaptation (DA) attempts to transfer the knowledge from a labeled source domain to an unlabeled target domain that follows different distribution from the source. To achieve this, DA methods include a source classification objective to extract the source knowledge and a domain alignment objective to diminish the domain shift, ensuring knowledge transfer. Typically, former DA methods adopt some weight hyper-parameters to linearly combine the training objectives to form an overall objective. However, the gradient directions of these objectives may conflict with each other due to domain shift. Under such circumstances, the linear optimization scheme might decrease the overall objective value at the expense of damaging one of the training objectives, leading to restricted solutions. In this paper, we rethink the optimization scheme for DA from a gradient-based perspective. We propose a Pareto Domain Adaptation (ParetoDA) approach to control the overall optimization direction, aiming to cooperatively optimize all training objectives. Specifically, to reach a desirable solution on the target domain, we design a surrogate loss mimicking target classification. To improve target-prediction accuracy to support the mimicking, we propose a target-prediction refining mechanism which exploits domain labels via Bayes' theorem. On the other hand, since prior knowledge of weighting schemes for objectives is often unavailable to guide optimization to approach the optimal solution on the target domain, we propose a dynamic preference mechanism to dynamically guide our cooperative optimization by the gradient of the surrogate loss on a held-out unlabeled target dataset. Extensive experiments on image classification and semantic segmentation benchmarks demonstrate the effectiveness of ParetoDA

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/24/2022

Source-Free Domain Adaptation via Distribution Estimation

Domain Adaptation aims to transfer the knowledge learned from a labeled ...
research
09/01/2022

Towards Optimization and Model Selection for Domain Generalization: A Mixup-guided Solution

The distribution shifts between training and test data typically undermi...
research
05/23/2023

Pulling Target to Source: A New Perspective on Domain Adaptive Semantic Segmentation

Domain adaptive semantic segmentation aims to transfer knowledge from a ...
research
03/20/2021

Your Classifier can Secretly Suffice Multi-Source Domain Adaptation

Multi-Source Domain Adaptation (MSDA) deals with the transfer of task kn...
research
05/08/2020

Sparsely-Labeled Source Assisted Domain Adaptation

Domain Adaptation (DA) aims to generalize the classifier learned from th...
research
02/09/2023

Bridging the Sim2Real gap with CARE: Supervised Detection Adaptation with Conditional Alignment and Reweighting

Sim2Real domain adaptation (DA) research focuses on the constrained sett...
research
12/26/2019

Domain Adaptation Regularization for Spectral Pruning

Deep Neural Networks (DNNs) have recently been achieving state-of-the-ar...

Please sign up or login with your details

Forgot password? Click here to reset