UniAP: Towards Universal Animal Perception in Vision via Few-shot Learning

by   Meiqi Sun, et al.

Animal visual perception is an important technique for automatically monitoring animal health, understanding animal behaviors, and assisting animal-related research. However, it is challenging to design a deep learning-based perception model that can freely adapt to different animals across various perception tasks, due to the varying poses of a large diversity of animals, lacking data on rare species, and the semantic inconsistency of different tasks. We introduce UniAP, a novel Universal Animal Perception model that leverages few-shot learning to enable cross-species perception among various visual tasks. Our proposed model takes support images and labels as prompt guidance for a query image. Images and labels are processed through a Transformer-based encoder and a lightweight label encoder, respectively. Then a matching module is designed for aggregating information between prompt guidance and the query image, followed by a multi-head label decoder to generate outputs for various tasks. By capitalizing on the shared visual characteristics among different animals and tasks, UniAP enables the transfer of knowledge from well-studied species to those with limited labeled data or even unseen species. We demonstrate the effectiveness of UniAP through comprehensive experiments in pose estimation, segmentation, and classification tasks on diverse animal species, showcasing its ability to generalize and adapt to new classes with minimal labeled examples.


page 1

page 3

page 5


Joint Intermodal and Intramodal Label Transfers for Extremely Rare or Unseen Classes

In this paper, we present a label transfer model from texts to images fo...

Universal Few-shot Learning of Dense Prediction Tasks with Visual Token Matching

Dense prediction tasks are a fundamental class of problems in computer v...

Unifying Visual Perception by Dispersible Points Learning

We present a conceptually simple, flexible, and universal visual percept...

Semantic-Based Few-Shot Learning by Interactive Psychometric Testing

Few-shot classification tasks aim to classify images in query sets based...

tSF: Transformer-based Semantic Filter for Few-Shot Learning

Few-Shot Learning (FSL) alleviates the data shortage challenge via embed...

One-Shot Learning with Triplet Loss for Vegetation Classification Tasks

Triplet loss function is one of the options that can significantly impro...

PhenoBench – A Large Dataset and Benchmarks for Semantic Image Interpretation in the Agricultural Domain

The production of food, feed, fiber, and fuel is a key task of agricultu...

Please sign up or login with your details

Forgot password? Click here to reset