Stacked Semantic-Guided Network for Zero-Shot Sketch-Based Image Retrieval

by   Hao Wang, et al.

Zero-shot sketch-based image retrieval (ZS-SBIR) is a task of cross-domain image retrieval from a natural image gallery with free-hand sketch under a zero-shot scenario. Previous works mostly focus on a generative approach that takes a highly abstract and sparse sketch as input and then synthesizes the corresponding natural image. However, the intrinsic visual sparsity and large intra-class variance of the sketch make the learning of the conditional decoder more difficult and hence achieve unsatisfactory retrieval performance. In this paper, we propose a novel stacked semantic-guided network to address the unique characteristics of sketches in ZS-SBIR. Specifically, we devise multi-layer feature fusion networks that incorporate different intermediate feature representation information in a deep neural network to alleviate the intrinsic sparsity of sketches. In order to improve visual knowledge transfer from seen to unseen classes, we elaborate a coarse-to-fine conditional decoder that generates coarse-grained category-specific corresponding features first (taking auxiliary semantic information as conditional input) and then generates fine-grained instance-specific corresponding features (taking sketch representation as conditional input). Furthermore, regression loss and classification loss are utilized to preserve the semantic and discriminative information of the synthesized features respectively. Extensive experiments on the large-scale Sketchy dataset and TU-Berlin dataset demonstrate that our proposed approach outperforms state-of-the-art methods by more than 20% in retrieval performance.


page 4

page 9

page 10

page 11


Semantic Adversarial Network for Zero-Shot Sketch-Based Image Retrieval

Zero-shot sketch-based image retrieval (ZS-SBIR) is a specific cross-mod...

A Zero-Shot Framework for Sketch-based Image Retrieval

Sketch-based image retrieval (SBIR) is the task of retrieving images fro...

Large-Scale Historical Watermark Recognition: dataset and a new consistency-based approach

Historical watermark recognition is a highly practical, yet unsolved cha...

Learning Large Euclidean Margin for Sketch-based Image Retrieval

This paper addresses the problem of Sketch-Based Image Retrieval (SBIR),...

Progressive Domain-Independent Feature Decomposition Network for Zero-Shot Sketch-Based Image Retrieval

Zero-shot sketch-based image retrieval (ZS-SBIR) is a specific cross-mod...

Stacked Adversarial Network for Zero-Shot Sketch based Image Retrieval

Conventional approaches to Sketch-Based Image Retrieval (SBIR) assume th...

Bi-Directional Domain Translation for Zero-Shot Sketch-Based Image Retrieval

The goal of Sketch-Based Image Retrieval (SBIR) is using free-hand sketc...

Please sign up or login with your details

Forgot password? Click here to reset