Visual and Semantic Prototypes-Jointly Guided CNN for Generalized Zero-shot Learning

by   Chuanxing Geng, et al.

In the process of exploring the world, the curiosity constantly drives humans to cognize new things. Supposing you are a zoologist, for a presented animal image, you can recognize it immediately if you know its class. Otherwise, you would more likely attempt to cognize it by exploiting the side-information (e.g., semantic information, etc.) you have accumulated. Inspired by this, this paper decomposes the generalized zero-shot learning (G-ZSL) task into an open set recognition (OSR) task and a zero-shot learning (ZSL) task, where OSR recognizes seen classes (if we have seen (or known) them) and rejects unseen classes (if we have never seen (or known) them before), while ZSL identifies the unseen classes rejected by the former. Simultaneously, without violating OSR's assumptions (only known class knowledge is available in training), we also first attempt to explore a new generalized open set recognition (G-OSR) by introducing the accumulated side-information from known classes to OSR. For G-ZSL, such a decomposition essentially solves the class overfitting problem with easily misclassifying unseen classes as seen classes. The problem is ubiquitous in most existing G-ZSL methods. On the other hand, for G-OSR, introducing such semantic information of known classes not only improves the recognition performance but also endows OSR with the cognitive ability of unknown classes. Specifically, a visual and semantic prototypes-jointly guided convolutional neural network (VSG-CNN) is proposed to fulfill these two tasks (G-ZSL and G-OSR) in a unified end-to-end learning framework. Extensive experiments on benchmark datasets demonstrate the advantages of our learning framework.


page 1

page 8


Global Semantic Consistency for Zero-Shot Learning

In image recognition, there are many cases where training samples cannot...

From Classical to Generalized Zero-Shot Learning: a Simple Adaptation Process

Zero-shot learning (ZSL) is concerned with the recognition of previously...

A Prototype-Based Generalized Zero-Shot Learning Framework for Hand Gesture Recognition

Hand gesture recognition plays a significant role in human-computer inte...

Prototypical Priors: From Improving Classification to Zero-Shot Learning

Recent works on zero-shot learning make use of side information such as ...

Counterfactual Zero-Shot and Open-Set Visual Recognition

We present a novel counterfactual framework for both Zero-Shot Learning ...

An Entropy-guided Reinforced Partial Convolutional Network for Zero-Shot Learning

Zero-Shot Learning (ZSL) aims to transfer learned knowledge from observe...

Learning to Separate Domains in Generalized Zero-Shot and Open Set Learning: a probabilistic perspective

This paper studies the problem of domain division problem which aims to ...

Please sign up or login with your details

Forgot password? Click here to reset