Comprehensive Information Integration Modeling Framework for Video Titling

by   Shengyu Zhang, et al.
Zhejiang University
Alibaba Group

In e-commerce, consumer-generated videos, which in general deliver consumers' individual preferences for the different aspects of certain products, are massive in volume. To recommend these videos to potential consumers more effectively, diverse and catchy video titles are critical. However, consumer-generated videos seldom accompany appropriate titles. To bridge this gap, we integrate comprehensive sources of information, including the content of consumer-generated videos, the narrative comment sentences supplied by consumers, and the product attributes, in an end-to-end modeling framework. Although automatic video titling is very useful and demanding, it is much less addressed than video captioning. The latter focuses on generating sentences that describe videos as a whole while our task requires the product-aware multi-grained video analysis. To tackle this issue, the proposed method consists of two processes, i.e., granular-level interaction modeling and abstraction-level story-line summarization. Specifically, the granular-level interaction modeling first utilizes temporal-spatial landmark cues, descriptive words, and abstractive attributes to builds three individual graphs and recognizes the intra-actions in each graph through Graph Neural Networks (GNN). Then the global-local aggregation module is proposed to model inter-actions across graphs and aggregate heterogeneous graphs into a holistic graph representation. The abstraction-level story-line summarization further considers both frame-level video features and the holistic graph to utilize the interactions between products and backgrounds, and generate the story-line topic of the video. We collect a large-scale dataset accordingly from real-world data in Taobao, a world-leading e-commerce platform, and will make the desensitized version publicly available to nourish further development of the research community...


page 2

page 4


Poet: Product-oriented Video Captioner for E-commerce

In e-commerce, a growing number of user-generated videos are used for pr...

Holistic Large Scale Video Understanding

Action recognition has been advanced in recent years by benchmarks with ...

Recipe Generation from Unsegmented Cooking Videos

This paper tackles recipe generation from unsegmented cooking videos, a ...

Leveraging Tripartite Interaction Information from Live Stream E-Commerce for Improving Product Recommendation

Recently, a new form of online shopping becomes more and more popular, w...

TruNet: Short Videos Generation from Long Videos via Story-Preserving Truncation

In this work, we introduce a new problem, named as story-preserving lon...

Landmark Enhanced Multimodal Graph Learning for Deepfake Video Detection

With the rapid development of face forgery technology, deepfake videos h...

E-commerce Recommendation with Weighted Expected Utility

Different from shopping at retail stores, consumers on e-commerce platfo...

Please sign up or login with your details

Forgot password? Click here to reset