Domain Disentangled Generative Adversarial Network for Zero-Shot Sketch-Based 3D Shape Retrieval

by   Rui Xu, et al.

Sketch-based 3D shape retrieval is a challenging task due to the large domain discrepancy between sketches and 3D shapes. Since existing methods are trained and evaluated on the same categories, they cannot effectively recognize the categories that have not been used during training. In this paper, we propose a novel domain disentangled generative adversarial network (DD-GAN) for zero-shot sketch-based 3D retrieval, which can retrieve the unseen categories that are not accessed during training. Specifically, we first generate domain-invariant features and domain-specific features by disentangling the learned features of sketches and 3D shapes, where the domain-invariant features are used to align with the corresponding word embeddings. Then, we develop a generative adversarial network that combines the domainspecific features of the seen categories with the aligned domain-invariant features to synthesize samples, where the synthesized samples of the unseen categories are generated by using the corresponding word embeddings. Finally, we use the synthesized samples of the unseen categories combined with the real samples of the seen categories to train the network for retrieval, so that the unseen categories can be recognized. In order to reduce the domain shift between the synthesized domain and the real domain, we adopt the transductive setting to reduce the gap between the distributions of the synthesized unseen categories and real unseen categories. Extensive experiments on the SHREC'13 and SHREC'14 datasets show that our method significantly improves the retrieval performance of the unseen categories.


Bi-Directional Domain Translation for Zero-Shot Sketch-Based Image Retrieval

The goal of Sketch-Based Image Retrieval (SBIR) is using free-hand sketc...

Alleviating Feature Confusion for Generative Zero-shot Learning

Lately, generative adversarial networks (GANs) have been successfully ap...

Three-Stream Joint Network for Zero-Shot Sketch-Based Image Retrieval

The Zero-Shot Sketch-based Image Retrieval (ZS-SBIR) is a challenging ta...

Airfoil GAN: Encoding and Synthesizing Airfoils forAerodynamic-aware Shape Optimization

The current design of aerodynamic shapes, like airfoils, involves comput...

Bi-Semantic Reconstructing Generative Network for Zero-shot Learning

Many recent methods of zero-shot learning (ZSL) attempt to utilize gener...

Semantically Tied Paired Cycle Consistency for Any-Shot Sketch-based Image Retrieval

Low-shot sketch-based image retrieval is an emerging task in computer vi...

ProtoGAN: Towards Few Shot Learning for Action Recognition

Few-shot learning (FSL) for action recognition is a challenging task of ...

Please sign up or login with your details

Forgot password? Click here to reset