Multi-label classification is an essential task utilized in a wide variety of
real-world applications. Multi-label zero-shot learning is a method for
classifying images into multiple unseen categories for which no training data
is available, while in general zero-shot situations, the test set may include
observed classes. The CLIP-Decoder is a novel method based on the
state-of-the-art ML-Decoder attention-based head. We introduce multi-modal
representation learning in CLIP-Decoder, utilizing the text encoder to extract
text features and the image encoder for image feature extraction. Furthermore,
we minimize semantic mismatch by aligning image and word embeddings in the same
dimension and comparing their respective representations using a combined loss,
which comprises classification loss and CLIP loss. This strategy outperforms
other methods and we achieve cutting-edge results on zero-shot multilabel
classification tasks using CLIP-Decoder. Our method achieves an absolute
increase of 3.9% in performance compared to existing methods for zero-shot
learning multi-label classification tasks. Additionally, in the generalized
zero-shot learning multi-label classification task, our method shows an
impressive increase of almost 2.3%.

多标签分类的零样本学习方法，通过引入 CLIP-Decoder 的多模态表示学习，在零样本多标签分类任务中表现出前沿结果，相较于现有方法在性能上提升了 3.9％，在广义零样本多标签分类任务中提升了近 2.3％。