Fine-Grained Representation Learning and Recognition by Exploiting Hierarchical Semantic Embedding

by   Tianshui Chen, et al.

Object categories inherently form a hierarchy with different levels of concept abstraction, especially for fine-grained categories. For example, birds (Aves) can be categorized according to a four-level hierarchy of order, family, genus, and species. This hierarchy encodes rich correlations among various categories across different levels, which can effectively regularize the semantic space and thus make prediction less ambiguous. However, previous studies of fine-grained image recognition primarily focus on categories of one certain level and usually overlook this correlation information. In this work, we investigate simultaneously predicting categories of different levels in the hierarchy and integrating this structured correlation information into the deep neural network by developing a novel Hierarchical Semantic Embedding (HSE) framework. Specifically, the HSE framework sequentially predicts the category score vector of each level in the hierarchy, from highest to lowest. At each level, it incorporates the predicted score vector of the higher level as prior knowledge to learn finer-grained feature representation. During training, the predicted score vector of the higher level is also employed to regularize label prediction by using it as soft targets of corresponding sub-categories. To evaluate the proposed framework, we organize the 200 bird species of the Caltech-UCSD birds dataset with the four-level category hierarchy and construct a large-scale butterfly dataset that also covers four level categories. Extensive experiments on these two and the newly-released VegFru datasets demonstrate the superiority of our HSE framework over the baseline methods and existing competitors.


page 3

page 5

page 7


Adaptive Semantic-Visual Tree for Hierarchical Embeddings

Merchandise categories inherently form a semantic hierarchy with differe...

Knowledge-Embedded Representation Learning for Fine-Grained Image Recognition

Humans can naturally understand an image in depth with the aid of rich k...

Label Hierarchy Transition: Modeling Class Hierarchies to Enhance Deep Classifiers

Hierarchical classification aims to sort the object into a hierarchy of ...

Embedding Label Structures for Fine-Grained Feature Representation

Recent algorithms in convolutional neural networks (CNN) considerably ad...

Semantic Guided Level-Category Hybrid Prediction Network for Hierarchical Image Classification

Hierarchical classification (HC) assigns each object with multiple label...

ConceptVision: A Flexible Scene Classification Framework

We introduce ConceptVision, a method that aims for high accuracy in cate...

SMTNet: Hierarchical cavitation intensity recognition based on sub-main transfer network

With the rapid development of smart manufacturing, data-driven machinery...

Please sign up or login with your details

Forgot password? Click here to reset