An Efficient Framework for Zero-Shot Sketch-Based Image Retrieval

by   Osman Tursun, et al.

Recently, Zero-shot Sketch-based Image Retrieval (ZS-SBIR) has attracted the attention of the computer vision community due to it's real-world applications, and the more realistic and challenging setting than found in SBIR. ZS-SBIR inherits the main challenges of multiple computer vision problems including content-based Image Retrieval (CBIR), zero-shot learning and domain adaptation. The majority of previous studies using deep neural networks have achieved improved results through either projecting sketch and images into a common low-dimensional space or transferring knowledge from seen to unseen classes. However, those approaches are trained with complex frameworks composed of multiple deep convolutional neural networks (CNNs) and are dependent on category-level word labels. This increases the requirements on training resources and datasets. In comparison, we propose a simple and efficient framework that does not require high computational training resources, and can be trained on datasets without semantic categorical labels. Furthermore, at training and inference stages our method only uses a single CNN. In this work, a pre-trained ImageNet CNN (e.g., ResNet50) is fine-tuned with three proposed learning objects: domain-aware quadruplet loss, semantic classification loss, and semantic knowledge preservation loss. The domain-aware quadruplet and semantic classification losses are introduced to learn discriminative, semantic and domain invariant features through considering ZS-SBIR as object detection and verification problem. ...


page 2

page 4

page 8

page 10


Semantic-Aware Knowledge Preservation for Zero-Shot Sketch-Based Image Retrieval

Sketch-based image retrieval (SBIR) is widely recognized as an important...

Sketch-an-Anchor: Sub-epoch Fast Model Adaptation for Zero-shot Sketch-based Image Retrieval

Sketch-an-Anchor is a novel method to train state-of-the-art Zero-shot S...

Freehand Sketch Recognition Using Deep Features

Freehand sketches often contain sparse visual detail. In spite of the sp...

Compact and Effective Representations for Sketch-based Image Retrieval

Sketch-based image retrieval (SBIR) has undergone an increasing interest...

WAD-CMSN: Wasserstein Distance based Cross-Modal Semantic Network for Zero-Shot Sketch-Based Image Retrieval

Zero-shot sketch-based image retrieval (ZSSBIR), as a popular studied br...

BDA-SketRet: Bi-Level Domain Adaptation for Zero-Shot SBIR

The efficacy of zero-shot sketch-based image retrieval (ZS-SBIR) models ...

Sketch-based 3D Shape Retrieval using Convolutional Neural Networks

Retrieving 3D models from 2D human sketches has received considerable at...

Please sign up or login with your details

Forgot password? Click here to reset