SlimFL: Federated Learning with Superposition Coding over Slimmable Neural Networks

by   Won Joon Yun, et al.

Federated learning (FL) is a key enabler for efficient communication and computing leveraging devices' distributed computing capabilities. However, applying FL in practice is challenging due to the local devices' heterogeneous energy, wireless channel conditions, and non-independently and identically distributed (non-IID) data distributions. To cope with these issues, this paper proposes a novel learning framework by integrating FL and width-adjustable slimmable neural networks (SNN). Integrating FL with SNNs is challenging due to time-varing channel conditions and data distributions. In addition, existing multi-width SNN training algorithms are sensitive to the data distributions across devices, which makes SNN ill-suited for FL. Motivated by this, we propose a communication and energy-efficient SNN-based FL (named SlimFL) that jointly utilizes superposition coding (SC) for global model aggregation and superposition training (ST) for updating local models. By applying SC, SlimFL exchanges the superposition of multiple width configurations decoded as many times as possible for a given communication throughput. Leveraging ST, SlimFL aligns the forward propagation of different width configurations while avoiding inter-width interference during backpropagation. We formally prove the convergence of SlimFL. The result reveals that SlimFL is not only communication-efficient but also deals with the non-IID data distributions and poor channel conditions, which is also corroborated by data-intensive simulations.


page 1

page 3

page 16


Joint Superposition Coding and Training for Federated Learning over Multi-Width Neural Networks

This paper aims to integrate two synergetic technologies, federated lear...

Communication and Energy Efficient Slimmable Federated Learning via Superposition Coding and Successive Decoding

Mobile devices are indispensable sources of big data. Federated learning...

FedNC: A Secure and Efficient Federated Learning Method Inspired by Network Coding

Federated Learning (FL) is a promising distributed learning mechanism wh...

Characterization of the Global Bias Problem in Aerial Federated Learning

Unmanned aerial vehicles (UAVs) mobility enables flexible and customized...

Semi-Synchronous Federated Learning

There are situations where data relevant to a machine learning problem a...

Distributed Learning Meets 6G: A Communication and Computing Perspective

With the ever-improving computing capabilities and storage capacities of...

Revisiting Cellular Throughput Prediction: Learning in-situ for Multi-device and Multi-network Considerations for 5G

Recent advancement in the ultra-highspeed cellular networks has pivoted ...

Please sign up or login with your details

Forgot password? Click here to reset