Invariance Analysis of Saliency Models versus Human Gaze During Scene Free Viewing

by   Zhaohui Che, et al.

Most of current studies on human gaze and saliency modeling have used high-quality stimuli. In real world, however, captured images undergo various types of distortions during the whole acquisition, transmission, and displaying chain. Some distortion types include motion blur, lighting variations and rotation. Despite few efforts, influences of ubiquitous distortions on visual attention and saliency models have not been systematically investigated. In this paper, we first create a large-scale database including eye movements of 10 observers over 1900 images degraded by 19 types of distortions. Second, by analyzing eye movements and saliency models, we find that: a) observers look at different locations over distorted versus original images, and b) performances of saliency models are drastically hindered over distorted images, with the maximum performance drop belonging to Rotation and Shearing distortions. Finally, we investigate the effectiveness of different distortions when serving as data augmentation transformations. Experimental results verify that some useful data augmentation transformations which preserve human gaze of reference images can improve deep saliency models against distortions, while some invalid transformations which severely change human gaze will degrade the performance.


page 5

page 7

page 9

page 11

page 12

page 14


GazeGAN: A Generative Adversarial Saliency Model based on Invariance Analysis of Human Gaze During Scene Free Viewing

Data size is the bottleneck for developing deep saliency models, because...

Leverage eye-movement data for saliency modeling: Invariance Analysis and a Robust New Model

Data size is the bottleneck for developing deep saliency models, because...

Fixation prediction with a combined model of bottom-up saliency and vanishing point

By predicting where humans look in natural scenes, we can understand how...

Improving saliency models' predictions of the next fixation with humans' intrinsic cost of gaze shifts

The human prioritization of image regions can be modeled in a time invar...

GASP: Gated Attention For Saliency Prediction

Saliency prediction refers to the computational task of modeling overt a...

The Effect of Distortions on the Prediction of Visual Attention

Existing saliency models have been designed and evaluated for predicting...

Toward Improving the Evaluation of Visual Attention Models: a Crowdsourcing Approach

Human visual attention is a complex phenomenon. A computational modeling...

Please sign up or login with your details

Forgot password? Click here to reset