Domain Adaptation Regularization for Spectral Pruning

by   Laurent Dillard, et al.

Deep Neural Networks (DNNs) have recently been achieving state-of-the-art performance on a variety of computer vision related tasks. However, their computational cost limits their ability to be implemented in embedded systems with restricted resources or strict latency constraints. Model compression has therefore been an active field of research to overcome this issue. On the other hand, DNNs typically require massive amounts of labeled data to be trained. This represents a second limitation to their deployment. Domain Adaptation (DA) addresses this issue by allowing to transfer knowledge learned on one labeled source distribution to a target distribution, possibly unlabeled. In this paper, we investigate on possible improvements of compression methods in DA setting. We focus on a compression method that was previously developed in the context of a single data distribution and show that, with a careful choice of data to use during compression and additional regularization terms directly related to DA objectives, it is possible to improve compression results. We also show that our method outperforms an existing compression method studied in the DA setting by a large margin for high compression rates. Although our work is based on one specific compression method, we also outline some general guidelines for improving compression in DA setting.


page 1

page 2

page 3

page 4


Biased Bagging for Unsupervised Domain Adaptation

Unsupervised domain adaptation (DA) is an active research area whose dev...

Deep Visual Domain Adaptation

Domain adaptation (DA) aims at improving the performance of a model on t...

A Compact DNN: Approaching GoogLeNet-Level Accuracy of Classification and Domain Adaptation

Recently, DNN model compression based on network architecture design, e....

Domain Adaptation by Topology Regularization

Deep learning has become the leading approach to assisted target recogni...

Balancing Discriminability and Transferability for Source-Free Domain Adaptation

Conventional domain adaptation (DA) techniques aim to improve domain tra...

Pareto Domain Adaptation

Domain adaptation (DA) attempts to transfer the knowledge from a labeled...

Domain Adaptation of Majority Votes via Perturbed Variation-based Label Transfer

We tackle the PAC-Bayesian Domain Adaptation (DA) problem. This arrives ...

Please sign up or login with your details

Forgot password? Click here to reset