HarmoFL: Harmonizing Local and Global Drifts in Federated Learning on Heterogeneous Medical Images

by   Meirui Jiang, et al.

Multiple medical institutions collaboratively training a model using federated learning (FL) has become a promising solution for maximizing the potential of data-driven models, yet the non-independent and identically distributed (non-iid) data in medical images is still an outstanding challenge in real-world practice. The feature heterogeneity caused by diverse scanners or protocols introduces a drift in the learning process, in both local (client) and global (server) optimizations, which harms the convergence as well as model performance. Many previous works have attempted to address the non-iid issue by tackling the drift locally or globally, but how to jointly solve the two essentially coupled drifts is still unclear. In this work, we concentrate on handling both local and global drifts and introduce a new harmonizing framework called HarmoFL. First, we propose to mitigate the local update drift by normalizing amplitudes of images transformed into the frequency domain to mimic a unified imaging setting, in order to generate a harmonized feature space across local clients. Second, based on harmonized features, we design a client weight perturbation guiding each local model to reach a flat optimum, where a neighborhood area of the local optimal solution has a uniformly low loss. Without any extra communication cost, the perturbation assists the global model to optimize towards a converged optimal solution by aggregating several local flat optima. We have theoretically analyzed the proposed method and empirically conducted extensive experiments on three medical image classification and segmentation tasks, showing that HarmoFL outperforms a set of recent state-of-the-art methods with promising convergence behavior.


page 5

page 7

page 13

page 14


FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correction

Federated learning (FL) allows multiple clients to collectively train a ...

Rethinking Client Drift in Federated Learning: A Logit Perspective

Federated Learning (FL) enables multiple clients to collaboratively lear...

Improving the Model Consistency of Decentralized Federated Learning

To mitigate the privacy leakages and communication burdens of Federated ...

Federated Split BERT for Heterogeneous Text Classification

Pre-trained BERT models have achieved impressive performance in many nat...

Endoscopy Classification Model Using Swin Transformer and Saliency Map

Endoscopy is a valuable tool for the early diagnosis of colon cancer. Ho...

Robust Split Federated Learning for U-shaped Medical Image Networks

U-shaped networks are widely used in various medical image tasks, such a...

Cooperative Hardware-Prompt Learning for Snapshot Compressive Imaging

Snapshot compressive imaging emerges as a promising technology for acqui...

Please sign up or login with your details

Forgot password? Click here to reset