Multimodal Deep Learning Framework for Image Popularity Prediction on Social Media

by   Fatma S. Abousaleh, et al.

Billions of photos are uploaded to the web daily through various types of social networks. Some of these images receive millions of views and become popular, whereas others remain completely unnoticed. This raises the problem of predicting image popularity on social media. The popularity of an image can be affected by several factors, such as visual content, aesthetic quality, user, post metadata, and time. Thus, considering all these factors is essential for accurately predicting image popularity. In addition, the efficiency of the predictive model also plays a crucial role. In this study, motivated by multimodal learning, which uses information from various modalities, and the current success of convolutional neural networks (CNNs) in various fields, we propose a deep learning model, called visual-social convolutional neural network (VSCNN), which predicts the popularity of a posted image by incorporating various types of visual and social features into a unified network model. VSCNN first learns to extract high-level representations from the input visual and social features by utilizing two individual CNNs. The outputs of these two networks are then fused into a joint network to estimate the popularity score in the output layer. We assess the performance of the proposed method by conducting extensive experiments on a dataset of approximately 432K images posted on Flickr. The simulation results demonstrate that the proposed VSCNN model significantly outperforms state-of-the-art models, with a relative improvement of greater than 2.33 terms of Spearman's Rho, mean absolute error, and mean squared error, respectively.


page 1

page 4

page 8

page 9

page 10

page 14


A Multimodal Approach to Predict Social Media Popularity

Multiple modalities represent different aspects by which information is ...

Intrinsic Image Popularity Assessment

The goal of research in image popularity assessment (IPA) is to develop ...

Predicting Popularity of Images Over 30 Days

The current work deals with the problem of attempting to predict the pop...

Time Matters: Multi-scale Temporalization of Social Media Popularity

The evolution of social media popularity exhibits rich temporality, i.e....

Predicting the Popularity of Micro-videos with Multimodal Variational Encoder-Decoder Framework

As an emerging type of user-generated content, micro-video drastically e...

Deep Neural Networks In Fully Connected CRF For Image Labeling With Social Network Metadata

We propose a novel method for predicting image labels by fusing image co...

MONITOR: A Multimodal Fusion Framework to Assess Message Veracity in Social Networks

Users of social networks tend to post and share content with little rest...

Please sign up or login with your details

Forgot password? Click here to reset