FedSSC: Shared Supervised-Contrastive Federated Learning

01/14/2023
by   Sirui Hu, et al.
0

Federated learning is widely used to perform decentralized training of a global model on multiple devices while preserving the data privacy of each device. However, it suffers from heterogeneous local data on each training device which increases the difficulty to reach the same level of accuracy as the centralized training. Supervised Contrastive Learning which outperform cross-entropy tries to minimizes the difference between feature space of points belongs to the same class and pushes away points from different classes. We propose Supervised Contrastive Federated Learning in which devices can share the learned class-wise feature spaces with each other and add the supervised-contrastive learning loss as a regularization term to foster the feature space learning. The loss tries to minimize the cosine similarity distance between the feature map and the averaged feature map from another device in the same class and maximizes the distance between the feature map and that in a different class. This new regularization term when added on top of the moon regularization term is found to outperform the other state-of-the-art regularization terms in solving the heterogeneous data distribution problem.

READ FULL TEXT

page 1

page 2

page 3

page 4

06/02/2018

Federated Learning with Non-IID Data

Federated learning enables resource-constrained edge compute devices, su...
03/30/2021

Model-Contrastive Federated Learning

Federated learning enables multiple parties to collaboratively train a m...
11/17/2022

FedFA: Federated Learning with Feature Anchors to Align Feature and Classifier for Heterogeneous Data

Federated learning allows multiple clients to collaboratively train a mo...
04/01/2023

MP-FedCL: Multi-Prototype Federated Contrastive Learning for Edge Intelligence

Federated learning-assisted edge intelligence enables privacy protection...
06/18/2021

Zero-Shot Federated Learning with New Classes for Audio Classification

Federated learning is an effective way of extracting insights from diffe...
10/01/2022

Federated Representation Learning via Maximal Coding Rate Reduction

We propose a federated methodology to learn low-dimensional representati...

Please sign up or login with your details

Forgot password? Click here to reset