Open Set Recognition using Vision Transformer with an Additional Detection Head

by   Feiyang Cai, et al.

Deep neural networks have demonstrated prominent capacities for image classification tasks in a closed set setting, where the test data come from the same distribution as the training data. However, in a more realistic open set scenario, traditional classifiers with incomplete knowledge cannot tackle test data that are not from the training classes. Open set recognition (OSR) aims to address this problem by both identifying unknown classes and distinguishing known classes simultaneously. In this paper, we propose a novel approach to OSR that is based on the vision transformer (ViT) technique. Specifically, our approach employs two separate training stages. First, a ViT model is trained to perform closed set classification. Then, an additional detection head is attached to the embedded features extracted by the ViT, trained to force the representations of known data to class-specific clusters compactly. Test examples are identified as known or unknown based on their distance to the cluster centers. To the best of our knowledge, this is the first time to leverage ViT for the purpose of OSR, and our extensive evaluation against several OSR benchmark datasets reveals that our approach significantly outperforms other baseline methods and obtains new state-of-the-art performance.


page 1

page 2

page 3

page 4


Open-Set Recognition with Gradient-Based Representations

Neural networks for image classification tasks assume that any given ima...

Conditional Gaussian Distribution Learning for Open Set Recognition

Deep neural networks have achieved state-of-the-art performance in a wid...

C2AE: Class Conditioned Auto-Encoder for Open-set Recognition

Models trained for classification often assume that all testing classes ...

OpenGAN: Open-Set Recognition via Open Data Generation

Real-world machine learning systems need to analyze novel testing data t...

Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4

In this work, we carry out a data archaeology to infer books that are kn...

Association Graph Learning for Multi-Task Classification with Category Shifts

In this paper, we focus on multi-task classification, where related clas...

Evaluation of Various Open-Set Medical Imaging Tasks with Deep Neural Networks

The current generation of deep neural networks has achieved close-to-hum...

Please sign up or login with your details

Forgot password? Click here to reset