ViCo: Engaging Video Comment Generation with Human Preference Rewards

by   Yuchong Sun, et al.

Engaging video comments play an important role in video social media, as they are the carrier of feelings, thoughts, or humor of the audience. Preliminary works have made initial exploration for video comment generation by adopting caption-style encoder-decoder models. However, comment generation presents some unique challenges distinct from caption generation, which makes these methods somewhat less effective at generating engaging comments. In contrast to the objective and descriptive nature of captions, comments tend to be inherently subjective, making it hard to quantify and evaluate the engagement of comments. Furthermore, the scarcity of truly engaging comments brings difficulty to collecting enough high-quality training examples. In this paper, we propose ViCo with three novel designs to tackle the above challenges for generating engaging Video Comments. Firstly, to quantify the engagement of comments, we utilize the number of "likes" each comment receives as a proxy of human preference after an appropriate debiasing procedure. Secondly, to automatically evaluate the engagement of comments, we train a reward model to align its judgment to the above proxy. Our user studies indicate that this reward model effectively aligns with human judgments. Lastly, to alleviate the scarcity of high-quality comments, an initial generator is trained on readily available but noisy data to generate comments. Then the reward model is employed to offer feedback on the generated comments, thus optimizing the initial generator. To facilitate the research of video commenting, we collect a large video comment-dataset (ViCo-20k) with rich metadata from a popular video website. Experiments on ViCo-20k show that the comments generated by our ViCo model exhibit the best performance in terms of both quantitative and qualitative results, particularly when engagement is considered.


page 1

page 3

page 7


Live Video Comment Generation Based on Surrounding Frames and Live Comments

In this paper, we propose the task of live comment generation. Live comm...

Knowledge Enhanced Model for Live Video Comment Generation

Live video commenting is popular on video media platforms, as it can cre...

Top Comment or Flop Comment? Predicting and Explaining User Engagement in Online News Discussions

Comment sections below online news articles enjoy growing popularity amo...

Generating Pertinent and Diversified Comments with Topic-aware Pointer-Generator Networks

Comment generation, a new and challenging task in Natural Language Gener...

Multimodal Matching Transformer for Live Commenting

Automatic live commenting aims to provide real-time comments on videos f...

Automatic Article Commenting: the Task and Dataset

Comments of online articles provide extended views and improve user enga...

Towards Controlled and Diverse Generation of Article Comments

Much research in recent years has focused on automatic article commentin...

Please sign up or login with your details

Forgot password? Click here to reset