Finding the Topic of a Set of Images

by   Gonzalo Vaca-Castano, et al.

In this paper we introduce the problem of determining the topic that a set of images is describing, where every topic is represented as a set of words. Different from other problems like tag assignment or similar, a) we assume multiple images are used as input instead of single image, b) Input images are typically not visually related, c) Input images are not necessarily semantically close, and d) Output word space is unconstrained. In our proposed solution, visual information of each query image is used to retrieve similar images with text labels (tags) from an image database. We consider a scenario where the tags are very noisy and diverse, given that they were obtained by implicit crowd-sourcing in a database of 1 million images and over seventy seven thousand tags. The words or tags associated to each query are processed jointly in a word selection algorithm using random walks that allows to refine the search topic, rejecting words that are not part of the topic and produce a set of words that fairly describe the topic. Experiments on a dataset of 300 topics, with up to twenty images per topic, show that our algorithm performs better than the proposed baseline for any number of query images. We also present a new Conditional Random Field (CRF) word mapping algorithm that preserves the semantic similarity of the mapped words, increasing the performance of the results over the baseline.


Automatic Visual Theme Discovery from Joint Image and Text Corpora

A popular approach to semantic image understanding is to manually tag im...

Topic representation: finding more representative words in topic models

The top word list, i.e., the top-M words with highest marginal probabili...

Constructing Hierarchical Image-tags Bimodal Representations for Word Tags Alternative Choice

This paper describes our solution to the multi-modal learning challenge ...

Understanding Infographics through Textual and Visual Tag Prediction

We introduce the problem of visual hashtag discovery for infographics: e...

Retrieving Black-box Optimal Images from External Databases

Suppose we have a black-box function (e.g., deep neural network) that ta...

Hybrid Affinity Propagation

In this paper, we address a problem of managing tagged images with hybri...

A Multi-View Embedding Space for Modeling Internet Images, Tags, and their Semantics

This paper investigates the problem of modeling Internet images and asso...

Please sign up or login with your details

Forgot password? Click here to reset