Learning a Unified Embedding for Visual Search at Pinterest

by   Andrew Zhai, et al.

At Pinterest, we utilize image embeddings throughout our search and recommendation systems to help our users navigate through visual content by powering experiences like browsing of related content and searching for exact products for shopping. In this work we describe a multi-task deep metric learning system to learn a single unified image embedding which can be used to power our multiple visual search products. The solution we present not only allows us to train for multiple application objectives in a single deep neural network architecture, but takes advantage of correlated information in the combination of all training data from each application to generate a unified embedding that outperforms all specialized embeddings previously deployed for each product. We discuss the challenges of handling images from different domains such as camera photos, high quality web images, and clean product catalog images. We also detail how to jointly train for multiple product objectives and how to leverage both engagement data and human labeled data. In addition, our trained embeddings can also be binarized for efficient storage and retrieval without compromising precision and recall. Through comprehensive evaluations on offline metrics, user studies, and online A/B experiments, we demonstrate that our proposed unified embedding improves both relevance and engagement of our visual search products for both browsing and searching purposes when compared to existing specialized embeddings. Finally, the deployment of the unified embedding at Pinterest has drastically reduced the operation and engineering cost of maintaining multiple embeddings while improving quality.


page 1

page 3

page 7

page 9


ItemSage: Learning Product Embeddings for Shopping Recommendations at Pinterest

Learned embeddings for products are an important building block for web-...

One Embedding To Do Them All

Online shopping caters to the needs of millions of users daily. Search, ...

Shop The Look: Building a Large Scale Visual Shopping System at Pinterest

As online content becomes ever more visual, the demand for searching by ...

Uni-Retriever: Towards Learning The Unified Embedding Based Retriever in Bing Sponsored Search

Embedding based retrieval (EBR) is a fundamental building block in many ...

Reference Product Search

For a product of interest, we propose a search method to surface a set o...

Learning Unified Embedding for Apparel Recognition

In apparel recognition, specialized models (e.g. models trained for a pa...

Binary Embedding-based Retrieval at Tencent

Large-scale embedding-based retrieval (EBR) is the cornerstone of search...

Please sign up or login with your details

Forgot password? Click here to reset