Joint Optimization of Ranking and Calibration with Contextualized Hybrid Model

by   Xiang-Rong Sheng, et al.

Despite the development of ranking optimization techniques, the pointwise model remains the dominating approach for click-through rate (CTR) prediction. It can be attributed to the calibration ability of the pointwise model since the prediction can be viewed as the click probability. In practice, a CTR prediction model is also commonly assessed with the ranking ability, for which prediction models based on ranking losses (e.g., pairwise or listwise loss) usually achieve better performances than the pointwise loss. Previous studies have experimented with a direct combination of the two losses to obtain the benefit from both losses and observed an improved performance. However, previous studies break the meaning of output logit as the click-through rate, which may lead to sub-optimal solutions. To address this issue, we propose an approach that can Jointly optimize the Ranking and Calibration abilities (JRC for short). JRC improves the ranking ability by contrasting the logit value for the sample with different labels and constrains the predicted probability to be a function of the logit subtraction. We further show that JRC consolidates the interpretation of logits, where the logits model the joint distribution. With such an interpretation, we prove that JRC approximately optimizes the contextualized hybrid discriminative-generative objective. Experiments on public and industrial datasets and online A/B testing show that our approach improves both ranking and calibration abilities. Since May 2022, JRC has been deployed on the display advertising platform of Alibaba and has obtained significant performance improvements.


page 1

page 2

page 3

page 4


RankT5: Fine-Tuning T5 for Text Ranking with Ranking Losses

Recently, substantial progress has been made in text ranking based on pr...

Confidence Ranking for CTR Prediction

Model evolution and constant availability of data are two common phenome...

Non-parametric inference on calibration of predicted risks

Moderate calibration, the expected event probability among observations ...

An extensive empirical study of inconsistent labels in multi-version-project defect data sets

The label quality of defect data sets has a direct influence on the reli...

COLD: Towards the Next Generation of Pre-Ranking System

Multi-stage cascade architecture exists widely in many industrial system...

Capturing Conversion Rate Fluctuation during Sales Promotions: A Novel Historical Data Reuse Approach

Conversion rate (CVR) prediction is one of the core components in online...

Calibration Matters: Tackling Maximization Bias in Large-scale Advertising Recommendation Systems

Calibration is defined as the ratio of the average predicted click rate ...

Please sign up or login with your details

Forgot password? Click here to reset