DC-CCL: Device-Cloud Collaborative Controlled Learning for Large Vision Models

by   Yucheng Ding, et al.

Many large vision models have been deployed on the cloud for real-time services. Meanwhile, fresh samples are continuously generated on the served mobile device. How to leverage the device-side samples to improve the cloud-side large model becomes a practical requirement, but falls into the dilemma of no raw sample up-link and no large model down-link. Specifically, the user may opt out of sharing raw samples with the cloud due to the concern of privacy or communication overhead, while the size of some large vision models far exceeds the mobile device's runtime capacity. In this work, we propose a device-cloud collaborative controlled learning framework, called DC-CCL, enabling a cloud-side large vision model that cannot be directly deployed on the mobile device to still benefit from the device-side local samples. In particular, DC-CCL vertically splits the base model into two submodels, one large submodel for learning from the cloud-side samples and the other small submodel for learning from the device-side samples and performing device-cloud knowledge fusion. Nevertheless, on-device training of the small submodel requires the output of the cloud-side large submodel to compute the desired gradients. DC-CCL thus introduces a light-weight model to mimic the large cloud-side submodel with knowledge distillation, which can be offloaded to the mobile device to control its small submodel's optimization direction. Given the decoupling nature of two submodels in collaborative learning, DC-CCL also allows the cloud to take a pre-trained model and the mobile device to take another model with a different backbone architecture.


page 1

page 2

page 3

page 4


On-Device Learning with Cloud-Coordinated Data Augmentation for Extreme Model Personalization in Recommender Systems

Data heterogeneity is an intrinsic property of recommender systems, maki...

Device-Cloud Collaborative Learning for Recommendation

With the rapid development of storage and computing power on mobile devi...

DUET: A Tuning-Free Device-Cloud Collaborative Parameters Generation Framework for Efficient Device Model Generalization

Device Model Generalization (DMG) is a practical yet under-investigated ...

JointDNN: An Efficient Training and Inference Engine for Intelligent Mobile Cloud Computing Services

Deep neural networks are among the most influential architectures of dee...

IDEAL: Toward High-efficiency Device-Cloud Collaborative and Dynamic Recommendation System

Recommendation systems have shown great potential to solve the informati...

Model-Agnostic Decentralized Collaborative Learning for On-Device POI Recommendation

As an indispensable personalized service in Location-based Social Networ...

Device-Cloud Collaborative Recommendation via Meta Controller

On-device machine learning enables the lightweight deployment of recomme...

Please sign up or login with your details

Forgot password? Click here to reset