FashionSearchNet-v2: Learning Attribute Representations with Localization for Image Retrieval with Attribute Manipulation

by   Kenan E. Ak, et al.

The focus of this paper is on the problem of image retrieval with attribute manipulation. Our proposed work is able to manipulate the desired attributes of the query image while maintaining its other attributes. For example, the collar attribute of the query image can be changed from round to v-neck to retrieve similar images from a large dataset. A key challenge in e-commerce is that images have multiple attributes where users would like to manipulate and it is important to estimate discriminative feature representations for each of these attributes. The proposed FashionSearchNet-v2 architecture is able to learn attribute specific representations by leveraging on its weakly-supervised localization module, which ignores the unrelated features of attributes in the feature space, thus improving the similarity learning. The network is jointly trained with the combination of attribute classification and triplet ranking loss to estimate local representations. These local representations are then merged into a single global representation based on the instructed attribute manipulation where desired images can be retrieved with a distance metric. The proposed method also provides explainability for its retrieval process to help provide additional information on the attention of the network. Experiments performed on several datasets that are rich in terms of the number of attributes show that FashionSearchNet-v2 outperforms the other state-of-the-art attribute manipulation techniques. Different than our earlier work (FashionSearchNet), we propose several improvements in the learning procedure and show that the proposed FashionSearchNet-v2 can be generalized to different domains other than fashion.


page 2

page 5

page 10


Semi-supervised Feature-Level Attribute Manipulation for Fashion Image Retrieval

With a growing demand for the search by image, many works have studied t...

Supervised Attribute Information Removal and Reconstruction for Image Manipulation

The goal of attribute manipulation is to control specified attribute(s) ...

Let's Transfer Transformations of Shared Semantic Representations

With a good image understanding capability, can we manipulate the images...

Scalable Visual Attribute Extraction through Hidden Layers of a Residual ConvNet

Visual attributes play an essential role in real applications based on i...

Examining CNN Representations with respect to Dataset Bias

Given a pre-trained CNN without any testing samples, this paper proposes...

Leveraging Weakly Annotated Data for Fashion Image Retrieval and Label Prediction

In this paper, we present a method to learn a visual representation adap...

Cooperative Embeddings for Instance, Attribute and Category Retrieval

The goal of this paper is to retrieve an image based on instance, attrib...

Please sign up or login with your details

Forgot password? Click here to reset