Federated Zero-Shot Learning for Visual Recognition

by   Zhi Chen, et al.

Zero-shot learning is a learning regime that recognizes unseen classes by generalizing the visual-semantic relationship learned from the seen classes. To obtain an effective ZSL model, one may resort to curating training samples from multiple sources, which may inevitably raise the privacy concerns about data sharing across different organizations. In this paper, we propose a novel Federated Zero-Shot Learning FedZSL framework, which learns a central model from the decentralized data residing on edge devices. To better generalize to previously unseen classes, FedZSL allows the training data on each device sampled from the non-overlapping classes, which are far from the i.i.d. that traditional federated learning commonly assumes. We identify two key challenges in our FedZSL protocol: 1) the trained models are prone to be biased to the locally observed classes, thus failing to generalize to the unseen classes and/or seen classes appeared on other devices; 2) as each category in the training data comes from a single source, the central model is highly vulnerable to model replacement (backdoor) attacks. To address these issues, we propose three local objectives for visual-semantic alignment and cross-device alignment through relation distillation, which leverages the normalized class-wise covariance to regularize the consistency of the prediction logits across devices. To defend against the backdoor attacks, a feature magnitude defending technique is proposed. As malicious samples are less correlated to the given semantic attributes, the visual features of low magnitude will be discarded to stabilize model updates. The effectiveness and robustness of FedZSL are demonstrated by extensive experiments conducted on three zero-shot benchmark datasets.


Entropy-Based Uncertainty Calibration for Generalized Zero-Shot Learning

Compared to conventional zero-shot learning (ZSL) where recognising unse...

Semantic Disentangling Generalized Zero-Shot Learning

Generalized Zero-Shot Learning (GZSL) aims to recognize images from both...

Zero-Shot Learning Through Cross-Modal Transfer

This work introduces a model that can recognize objects in images even i...

Federated Few-Shot Learning with Adversarial Learning

We are interested in developing a unified machine learning model over ma...

Federated Zero-Shot Learning with Mid-Level Semantic Knowledge Transfer

Conventional centralised deep learning paradigms are not feasible when d...

FedZKT: Zero-Shot Knowledge Transfer towards Heterogeneous On-Device Models in Federated Learning

Federated learning enables distributed devices to collaboratively learn ...

ATZSL: Defensive Zero-Shot Recognition in the Presence of Adversaries

Zero-shot learning (ZSL) has received extensive attention recently espec...

Please sign up or login with your details

Forgot password? Click here to reset