Domain Adaptation Regularization for Spectral Pruning

12/26/2019
by   Laurent Dillard, et al.
11

Deep Neural Networks (DNNs) have recently been achieving state-of-the-art performance on a variety of computer vision related tasks. However, their computational cost limits their ability to be implemented in embedded systems with restricted resources or strict latency constraints. Model compression has therefore been an active field of research to overcome this issue. On the other hand, DNNs typically require massive amounts of labeled data to be trained. This represents a second limitation to their deployment. Domain Adaptation (DA) addresses this issue by allowing to transfer knowledge learned on one labeled source distribution to a target distribution, possibly unlabeled. In this paper, we investigate on possible improvements of compression methods in DA setting. We focus on a compression method that was previously developed in the context of a single data distribution and show that, with a careful choice of data to use during compression and additional regularization terms directly related to DA objectives, it is possible to improve compression results. We also show that our method outperforms an existing compression method studied in the DA setting by a large margin for high compression rates. Although our work is based on one specific compression method, we also outline some general guidelines for improving compression in DA setting.

READ FULL TEXT

page 1

page 2

page 3

page 4

06/16/2017

Biased Bagging for Unsupervised Domain Adaptation

Unsupervised domain adaptation (DA) is an active research area whose dev...
12/28/2020

Deep Visual Domain Adaptation

Domain adaptation (DA) aims at improving the performance of a model on t...
03/12/2017

A Compact DNN: Approaching GoogLeNet-Level Accuracy of Classification and Domain Adaptation

Recently, DNN model compression based on network architecture design, e....
01/28/2021

Domain Adaptation by Topology Regularization

Deep learning has become the leading approach to assisted target recogni...
06/16/2022

Balancing Discriminability and Transferability for Source-Free Domain Adaptation

Conventional domain adaptation (DA) techniques aim to improve domain tra...
12/08/2021

Pareto Domain Adaptation

Domain adaptation (DA) attempts to transfer the knowledge from a labeled...
11/19/2013

Domain Adaptation of Majority Votes via Perturbed Variation-based Label Transfer

We tackle the PAC-Bayesian Domain Adaptation (DA) problem. This arrives ...

Please sign up or login with your details

Forgot password? Click here to reset