Improving saliency models' predictions of the next fixation with humans' intrinsic cost of gaze shifts

by   Florian Kadner, et al.

The human prioritization of image regions can be modeled in a time invariant fashion with saliency maps or sequentially with scanpath models. However, while both types of models have steadily improved on several benchmarks and datasets, there is still a considerable gap in predicting human gaze. Here, we leverage two recent developments to reduce this gap: theoretical analyses establishing a principled framework for predicting the next gaze target and the empirical measurement of the human cost for gaze switches independently of image content. We introduce an algorithm in the framework of sequential decision making, which converts any static saliency map into a sequence of dynamic history-dependent value maps, which are recomputed after each gaze shift. These maps are based on 1) a saliency map provided by an arbitrary saliency model, 2) the recently measured human cost function quantifying preferences in magnitude and direction of eye movements, and 3) a sequential exploration bonus, which changes with each subsequent gaze shift. The parameters of the spatial extent and temporal decay of this exploration bonus are estimated from human gaze data. The relative contributions of these three components were optimized on the MIT1003 dataset for the NSS score and are sufficient to significantly outperform predictions of the next gaze target on NSS and AUC scores for five state of the art saliency models on three image data sets. Thus, we provide an implementation of human gaze preferences, which can be used to improve arbitrary saliency models' predictions of humans' next gaze targets.


page 6

page 8

page 19


Predicting Human Eye Fixations via an LSTM-based Saliency Attentive Model

Data-driven saliency has recently gained a lot of attention thanks to th...

Fixation prediction with a combined model of bottom-up saliency and vanishing point

By predicting where humans look in natural scenes, we can understand how...

Scanpath Prediction on Information Visualisations

We propose Unified Model of Saliency and Scanpaths (UMSS) – a model that...

Faster gaze prediction with dense networks and Fisher pruning

Predicting human fixations from images has recently seen large improveme...

How close are we to understanding image-based saliency?

Within the set of the many complex factors driving gaze placement, the p...

Invariance Analysis of Saliency Models versus Human Gaze During Scene Free Viewing

Most of current studies on human gaze and saliency modeling have used hi...

SalGaze: Personalizing Gaze Estimation Using Visual Saliency

Traditional gaze estimation methods typically require explicit user cali...

Please sign up or login with your details

Forgot password? Click here to reset