Federated Learning with Client-Exclusive Classes

by   Jiayun Zhang, et al.

Existing federated classification algorithms typically assume the local annotations at every client cover the same set of classes. In this paper, we aim to lift such an assumption and focus on a more general yet practical non-IID setting where every client can work on non-identical and even disjoint sets of classes (i.e., client-exclusive classes), and the clients have a common goal which is to build a global classification model to identify the union of these classes. Such heterogeneity in client class sets poses a new challenge: how to ensure different clients are operating in the same latent space so as to avoid the drift after aggregation? We observe that the classes can be described in natural languages (i.e., class names) and these names are typically safe to share with all parties. Thus, we formulate the classification problem as a matching process between data representations and class representations and break the classification model into a data encoder and a label encoder. We leverage the natural-language class names as the common ground to anchor the class representations in the label encoder. In each iteration, the label encoder updates the class representations and regulates the data representations through matching. We further use the updated class representations at each round to annotate data samples for locally-unaware classes according to similarity and distill knowledge to local models. Extensive experiments on four real-world datasets show that the proposed method can outperform various classical and state-of-the-art federated learning methods designed for learning with non-IID data.


Re-Weighted Softmax Cross-Entropy to Control Forgetting in Federated Learning

In Federated Learning, a global model is learned by aggregating model up...

FedABC: Targeting Fair Competition in Personalized Federated Learning

Federated learning aims to collaboratively train models without accessin...

Rethinking Client Drift in Federated Learning: A Logit Perspective

Federated Learning (FL) enables multiple clients to collaboratively lear...

Federated Unsupervised Representation Learning

To leverage enormous unlabeled data on distributed edge devices, we form...

Federated Learning with Classifier Shift for Class Imbalance

Federated learning aims to learn a global model collaboratively while th...

Federated Generalized Category Discovery

Generalized category discovery (GCD) aims at grouping unlabeled samples ...

Global Multiclass Classification from Heterogeneous Local Models

Multiclass classification problems are most often solved by either train...

Please sign up or login with your details

Forgot password? Click here to reset