Back-to-Bones: Rediscovering the Role of Backbones in Domain Generalization

by   Simone Angarano, et al.

Domain Generalization (DG) studies the capability of a deep learning model to generalize to out-of-training distributions. In the last decade, literature has been massively filled with a collection of training methodologies that claim to obtain more abstract and robust data representations to tackle domain shifts. Recent research has provided a reproducible benchmark for DG, pointing out the effectiveness of naive empirical risk minimization (ERM) over existing algorithms. Nevertheless, researchers persist in using the same outdated feature extractors, and no attention has been given to the effects of different backbones yet. In this paper, we start back to backbones proposing a comprehensive analysis of their intrinsic generalization capabilities, so far ignored by the research community. We evaluate a wide variety of feature extractors, from standard residual solutions to transformer-based architectures, finding an evident linear correlation between large-scale single-domain classification accuracy and DG capability. Our extensive experimentation shows that by adopting competitive backbones in conjunction with effective data augmentation, plain ERM outperforms recent DG solutions and achieves state-of-the-art accuracy. Moreover, our additional qualitative studies reveal that novel backbones give more similar representations to same-class samples, separating different domains in the feature space. This boost in generalization capabilities leaves marginal room for DG algorithms and suggests a new paradigm for investigating the problem, placing backbones in the spotlight and encouraging the development of consistent algorithms on top of them.


page 7

page 19

page 20


Adaptive Methods for Aggregated Domain Generalization

Domain generalization involves learning a classifier from a heterogeneou...

OoD-Bench: Benchmarking and Understanding Out-of-Distribution Generalization Datasets and Algorithms

Deep learning has achieved tremendous success with independent and ident...

Robust Representation Learning with Self-Distillation for Domain Generalization

Domain generalization is a challenging problem in machine learning, wher...

Randomized Adversarial Style Perturbations for Domain Generalization

We propose a novel domain generalization technique, referred to as Rando...

Localized Adversarial Domain Generalization

Deep learning methods can struggle to handle domain shifts not seen in t...

Single Domain Generalization via Normalised Cross-correlation Based Convolutions

Deep learning techniques often perform poorly in the presence of domain ...

Towards Principled Disentanglement for Domain Generalization

A fundamental challenge for machine learning models is generalizing to o...

Please sign up or login with your details

Forgot password? Click here to reset