EViT: Privacy-Preserving Image Retrieval via Encrypted Vision Transformer in Cloud Computing

by   Qihua Feng, et al.

Image retrieval systems help users to browse and search among extensive images in real-time. With the rise of cloud computing, retrieval tasks are usually outsourced to cloud servers. However, the cloud scenario brings a daunting challenge of privacy protection as cloud servers cannot be fully trusted. To this end, image-encryption-based privacy-preserving image retrieval schemes have been developed, which first extract features from cipher-images, and then build retrieval models based on these features. Yet, most existing approaches extract shallow features and design trivial retrieval models, resulting in insufficient expressiveness for the cipher-images. In this paper, we propose a novel paradigm named Encrypted Vision Transformer (EViT), which advances the discriminative representations capability of cipher-images. First, in order to capture comprehensive ruled information, we extract multi-level local length sequence and global Huffman-code frequency features from the cipher-images which are encrypted by stream cipher during JPEG compression process. Second, we design the Vision Transformer-based retrieval model to couple with the multi-level features, and propose two adaptive data augmentation methods to improve representation power of the retrieval model. Our proposal can be easily adapted to unsupervised and supervised settings via self-supervised contrastive learning manner. Extensive experiments reveal that EViT achieves both excellent encryption and retrieval performance, outperforming current schemes in terms of retrieval accuracy by large margins while protecting image privacy effectively. Code is publicly available at <https://github.com/onlinehuazai/EViT>.


MSPPIR: Multi-source privacy-preserving image retrieval in cloud computing

Content-Based Image Retrieval (CBIR) techniques have been widely researc...

Towards Privacy-Preserving, Real-Time and Lossless Feature Matching

Most visual retrieval applications store feature vectors for downstream ...

Content-Based Multi-Source Encrypted Image Retrieval in Clouds with Privacy Preservation

Content-based image retrieval (CBIR) is one of the fundamental image ret...

A Privacy-Preserving Image Retrieval Scheme with a Mixture of Plain and EtC Images

In this paper, we propose a novel content-based image-retrieval scheme t...

A Privacy-Preserving and End-to-End-Based Encrypted Image Retrieval Scheme

Applying encryption technology to image retrieval can ensure the securit...

Privacy-Preserving Image Retrieval Based on Additive Secret Sharing

The rapid growth of digital images motivates individuals and organizatio...

THE-X: Privacy-Preserving Transformer Inference with Homomorphic Encryption

As more and more pre-trained language models adopt on-cloud deployment, ...

Please sign up or login with your details

Forgot password? Click here to reset