Towards Optimization and Model Selection for Domain Generalization: A Mixup-guided Solution

by   Wang Lu, et al.
Institute of Computing Technology, Chinese Academy of Sciences

The distribution shifts between training and test data typically undermine the performance of deep learning models. In recent years, lots of work pays attention to domain generalization (DG) where distribution shift exists and target data are unseen. Despite the progress in algorithm design, two foundational factors have long been ignored: 1) the optimization for regularization-based objectives (e.g., distribution alignment), and 2) the model selection for DG since no knowledge about the target domain can be utilized. In this paper, we propose Mixup guided optimization and selection techniques for domain generalization. For optimization, we utilize an adapted Mixup to generate an out-of-distribution dataset that can guide the preference direction and optimize with Pareto optimization. For model selection, we generate a validation dataset with a closer distance to the target distribution, and thereby it can better represent the target data. We also present some theoretical insights behind our proposals. Comprehensive experiments on one visual classification benchmark and three time-series benchmarks demonstrate that our model optimization and selection techniques can largely improve the performance of existing domain generalization algorithms and even achieve new state-of-the-art results.


Pareto Domain Adaptation

Domain adaptation (DA) attempts to transfer the knowledge from a labeled...

A principled approach to model validation in domain generalization

Domain generalization aims to learn a model with good generalization abi...

Towards a Theoretical Framework of Out-of-Distribution Generalization

Generalization to out-of-distribution (OOD) data, or domain generalizati...

An Information-theoretic Approach to Distribution Shifts

Safely deploying machine learning models to the real world is often a ch...

Ensemble of Averages: Improving Model Selection and Boosting Performance in Domain Generalization

In Domain Generalization (DG) settings, models trained on a given set of...

Rich Feature Construction for the Optimization-Generalization Dilemma

There often is a dilemma between ease of optimization and robust out-of-...

Unseen Target Stance Detection with Adversarial Domain Generalization

Although stance detection has made great progress in the past few years,...

Please sign up or login with your details

Forgot password? Click here to reset