Animal visual perception is an important technique for automatically
monitoring animal health, understanding animal behaviors, and assisting
animal-related research. However, it is challenging to design a deep
learning-based perception model that can freely adapt to different animals
across various perception tasks, due to the varying poses of a large diversity
of animals, lacking data on rare species, and the semantic inconsistency of
different tasks. We introduce UniAP, a novel Universal Animal Perception model
that leverages few-shot learning to enable cross-species perception among
various visual tasks. Our proposed model takes support images and labels as
prompt guidance for a query image. Images and labels are processed through a
Transformer-based encoder and a lightweight label encoder, respectively. Then a
matching module is designed for aggregating information between prompt guidance
and the query image, followed by a multi-head label decoder to generate outputs
for various tasks. By capitalizing on the shared visual characteristics among
different animals and tasks, UniAP enables the transfer of knowledge from
well-studied species to those with limited labeled data or even unseen species.
We demonstrate the effectiveness of UniAP through comprehensive experiments in
pose estimation, segmentation, and classification tasks on diverse animal
species, showcasing its ability to generalize and adapt to new classes with
minimal labeled examples.

UniAP 是一种新型的通用动物感知模型，它利用少样本学习实现不同视觉任务中不同动物之间的跨物种感知。通过利用不同动物和任务之间的共享视觉特征，UniAP 能够将知识从研究充分的物种传递到有限标记数据甚至从未见过的物种中，展示其在姿势估计、分割和分类任务方面的广泛适应性和泛化能力。