Indicative Image Retrieval: Turning Blackbox Learning into Grey

by   Xulu Zhang, et al.

Deep learning became the game changer for image retrieval soon after it was introduced. It promotes the feature extraction (by representation learning) as the core of image retrieval, with the relevance/matching evaluation being degenerated into simple similarity metrics. In many applications, we need the matching evidence to be indicated rather than just have the ranked list (e.g., the locations of the target proteins/cells/lesions in medical images). It is like the matched words need to be highlighted in search engines. However, this is not easy to implement without explicit relevance/matching modeling. The deep representation learning models are not feasible because of their blackbox nature. In this paper, we revisit the importance of relevance/matching modeling in deep learning era with an indicative retrieval setting. The study shows that it is possible to skip the representation learning and model the matching evidence directly. By removing the dependency on the pre-trained models, it has avoided a lot of related issues (e.g., the domain gap between classification and retrieval, the detail-diffusion caused by convolution, and so on). More importantly, the study demonstrates that the matching can be explicitly modeled and backtracked later for generating the matching evidence indications. It can improve the explainability of deep inference. Our method obtains a best performance in literature on both Oxford-5k and Paris-6k, and sets a new record of 97.77 features.


page 2

page 3

page 4

page 5

page 6

page 8

page 10

page 11


Generating Binary Tags for Fast Medical Image Retrieval Based on Convolutional Nets and Radon Transform

Content-based image retrieval (CBIR) in large medical image archives is ...

Universal Perturbation Attack Against Image Retrieval

Despite the remarkable success, deep learning models have shown to be vu...

Ranking Loss and Sequestering Learning for Reducing Image Search Bias in Histopathology

Recently, deep learning has started to play an essential role in healthc...

Image retrieval with hierarchical matching pursuit

A novel representation of images for image retrieval is introduced in th...

Enhancing Dynamic Image Advertising with Vision-Language Pre-training

In the multimedia era, image is an effective medium in search advertisin...

A clinically motivated self-supervised approach for content-based image retrieval of CT liver images

Deep learning-based approaches for content-based image retrieval (CBIR) ...

Image Retrieval using Heat Diffusion for Deep Feature Aggregation

Image-level feature descriptors obtained from convolutional neural netwo...

Please sign up or login with your details

Forgot password? Click here to reset