Few-shot classification aims to adapt to new tasks with limited labeled
examples. To fully use the accessible data, recent methods explore suitable
measures for the similarity between the query and support images and better
high-dimensional features with meta-training and pre-training strategies.
However, the potential of multi-modality information has barely been explored,
which may bring promising improvement for few-shot classification. In this
paper, we propose a Language-guided Prototypical Network (LPN) for few-shot
classification, which leverages the complementarity of vision and language
modalities via two parallel branches. Concretely, to introduce language
modality with limited samples in the visual task, we leverage a pre-trained
text encoder to extract class-level text features directly from class names
while processing images with a conventional image encoder. Then, a
language-guided decoder is introduced to obtain text features corresponding to
each image by aligning class-level features with visual features. In addition,
to take advantage of class-level features and prototypes, we build a refined
prototypical head that generates robust prototypes in the text branch for
follow-up measurement. Finally, we aggregate the visual and text logits to
calibrate the deviation of a single modality. Extensive experiments demonstrate
the competitiveness of LPN against state-of-the-art methods on benchmark
datasets.

本文提出了一种基于语言引导的原型网络 (LPN)，通过引入语言模态和计算视觉特征向量之间的相似性，从而改进了少样本分类任务。实验结果表明，该方法在基准数据集上具有与最先进方法相当的竞争力。