Fine-Grained Object Recognition and Zero-Shot Learning in Remote Sensing Imagery

by   Gencer Sumbul, et al.

Fine-grained object recognition that aims to identify the type of an object among a large number of subcategories is an emerging application with the increasing resolution that exposes new details in image data. Traditional fully supervised algorithms fail to handle this problem where there is low between-class variance and high within-class variance for the classes of interest with small sample sizes. We study an even more extreme scenario named zero-shot learning (ZSL) in which no training example exists for some of the classes. ZSL aims to build a recognition model for new unseen categories by relating them to seen classes that were previously learned. We establish this relation by learning a compatibility function between image features extracted via a convolutional neural network and auxiliary information that describes the semantics of the classes of interest by using training samples from the seen classes. Then, we show how knowledge transfer can be performed for the unseen classes by maximizing this function during inference. We introduce a new data set that contains 40 different types of street trees in 1-ft spatial resolution aerial data, and evaluate the performance of this model with manually annotated attributes, a natural language model, and a scientific taxonomy as auxiliary information. The experiments show that the proposed model achieves 14.3 recognition accuracy for the classes with no training examples, which is significantly better than a random guess accuracy of 6.3 and three other ZSL algorithms.


page 1

page 9

page 10


Zero-Knowledge Zero-Shot Learning for Novel Visual Category Discovery

Generalized Zero-Shot Learning (GZSL) and Open-Set Recognition (OSR) are...

Convolutional Prototype Learning for Zero-Shot Recognition

Zero-shot learning (ZSL) has received increasing attention in recent yea...

Learning the Redundancy-free Features for Generalized Zero-Shot Object Recognition

Zero-shot object recognition or zero-shot learning aims to transfer the ...

Multisource Region Attention Network for Fine-Grained Object Recognition in Remote Sensing Imagery

Fine-grained object recognition concerns the identification of the type ...

ATZSL: Defensive Zero-Shot Recognition in the Presence of Adversaries

Zero-shot learning (ZSL) has received extensive attention recently espec...

Learning with Labels of Existing and Nonexisting

We study the classification or detection problems where the label only s...

CRL: Class Representative Learning for Image Classification

Building robust and real-time classifiers with diverse datasets are one ...

Please sign up or login with your details

Forgot password? Click here to reset