Cache-Augmented Inbatch Importance Resampling for Training Recommender Retriever

by   Jin Chen, et al.

Recommender retrievers aim to rapidly retrieve a fraction of items from the entire item corpus when a user query requests, with the representative two-tower model trained with the log softmax loss. For efficiently training recommender retrievers on modern hardwares, inbatch sampling, where the items in the mini-batch are shared as negatives to estimate the softmax function, has attained growing interest. However, existing inbatch sampling based strategies just correct the sampling bias of inbatch items with item frequency, being unable to distinguish the user queries within the mini-batch and still incurring significant bias from the softmax. In this paper, we propose a Cache-Augmented Inbatch Importance Resampling (XIR) for training recommender retrievers, which not only offers different negatives to user queries with inbatch items, but also adaptively achieves a more accurate estimation of the softmax distribution. Specifically, XIR resamples items for the given mini-batch training pairs based on certain probabilities, where a cache with more frequently sampled items is adopted to augment the candidate item set, with the purpose of reusing the historical informative samples. XIR enables to sample query-dependent negatives based on inbatch items and to capture dynamic changes of model training, which leads to a better approximation of the softmax and further contributes to better convergence. Finally, we conduct experiments to validate the superior performance of the proposed XIR compared with competitive approaches.


page 1

page 2

page 3

page 4


Cross-Batch Negative Sampling for Training Two-Tower Recommenders

The two-tower architecture has been widely applied for learning item and...

Distributed Online Learning in Social Recommender Systems

In this paper, we consider decentralized sequential decision making in d...

Represent Items by Items: An Enhanced Representation of the Target Item for Recommendation

Item-based collaborative filtering (ICF) has been widely used in industr...

On the Effectiveness of Sampled Softmax Loss for Item Recommendation

Learning objectives of recommender models remain largely unexplored. Mos...

Sampled Softmax with Random Fourier Features

The computational cost of training with softmax cross entropy loss grows...

Memory Augmented Multi-Instance Contrastive Predictive Coding for Sequential Recommendation

The sequential recommendation aims to recommend items, such as products,...

Exploiting Data Skew for Improved Query Performance

Analytic queries enable sophisticated large-scale data analysis within m...

Please sign up or login with your details

Forgot password? Click here to reset