Classifier Crafting: Turn Your ConvNet into a Zero-Shot Learner!

by   Jacopo Cavazza, et al.

In Zero-shot learning (ZSL), we classify unseen categories using textual descriptions about their expected appearance when observed (class embeddings) and a disjoint pool of seen classes, for which annotated visual data are accessible. We tackle ZSL by casting a "vanilla" convolutional neural network (e.g. AlexNet, ResNet-101, DenseNet-201 or DarkNet-53) into a zero-shot learner. We do so by crafting the softmax classifier: we freeze its weights using fixed seen classification rules, either semantic (seen class embeddings) or visual (seen class prototypes). Then, we learn a data-driven and ZSL-tailored feature representation on seen classes only to match these fixed classification rules. Given that the latter seamlessly generalize towards unseen classes, while requiring not actual unseen data to be computed, we can perform ZSL inference by augmenting the pool of classification rules at test time while keeping the very same representation we learnt: nowhere re-training or fine-tuning on unseen data is performed. The combination of semantic and visual crafting (by simply averaging softmax scores) improves prior state-of-the-art methods in benchmark datasets for standard, inductive ZSL. After rebalancing predictions to better handle the joint inference over seen and unseen classes, we outperform prior generalized, inductive ZSL methods as well. Also, we gain interpretability at no additional cost, by using neural attention methods (e.g., grad-CAM) as they are. Code will be made publicly available.


page 1

page 5

page 6

page 8


Meta-Transfer Networks for Zero-Shot Learning

Zero-Shot Learning (ZSL) aims at recognizing unseen categories using som...

Generalised Zero-Shot Learning with Domain Classification in a Joint Semantic and Visual Space

Generalised zero-shot learning (GZSL) is a classification problem where ...

Semantic Similarity Based Softmax Classifier for Zero-Shot Learning

Zero-Shot Learning (ZSL) is a classification task where we do not have e...

Generative Model-driven Structure Aligning Discriminative Embeddings for Transductive Zero-shot Learning

Zero-shot Learning (ZSL) is a transfer learning technique which aims at ...

Zero-shot Image Recognition Using Relational Matching, Adaptation and Calibration

Zero-shot learning (ZSL) for image classification focuses on recognizing...

Nearly Zero-Shot Learning for Semantic Decoding in Spoken Dialogue Systems

This paper presents two ways of dealing with scarce data in semantic dec...

One-Shot Learning in Discriminative Neural Networks

We consider the task of one-shot learning of visual categories. In this ...

Please sign up or login with your details

Forgot password? Click here to reset