Traditional bird classifiers mostly rely on the visual characteristics of
birds. Some prior works even train classifiers to be invariant to the
background, completely discarding the living environment of birds. Instead, we
are the first to explore integrating habitat information, one of the four major
cues for identifying birds by ornithologists, into modern bird classifiers. We
focus on two leading model types: (1) CNNs and ViTs trained on the downstream
bird datasets; and (2) original, multi-modal CLIP. Training CNNs and ViTs with
habitat-augmented data results in an improvement of up to +0.83 and +0.23
points on NABirds and CUB-200, respectively. Similarly, adding habitat
descriptors to the prompts for CLIP yields a substantial accuracy boost of up
to +0.99 and +1.1 points on NABirds and CUB-200, respectively. We find
consistent accuracy improvement after integrating habitat features into the
image augmentation process and into the textual descriptors of vision-language
CLIP classifiers. Code is available at:
this https URL

将鸟类分类器与栖息地信息集成，通过卷积神经网络（CNNs）、视觉注意力机制（ViTs）和多模态 CLIP 模型对鸟类数据集进行训练，通过整合栖息地特征改进分类器准确度。