Zero-shot learning methods rely on fixed visual and semantic embeddings,
extracted from independent vision and language models, both pre-trained for
other large-scale tasks. This is a weakness of current zero-shot learning
frameworks as such disjoint embeddings fail to adequately associate visual and
textual information to their shared semantic content. Therefore, we propose to
learn semantically grounded and enriched visual information by computing a
joint image and text model with a two-stream network on a proxy task. To
improve this alignment between image and textual representations, provided by
attributes, we leverage ancillary captions to provide grounded semantic
information. Our method, dubbed joint embeddings for zero-shot learning is
evaluated on several benchmark datasets, improving the performance of existing
state-of-the-art methods in both standard ($+1.6$\% on aPY, $+2.6\%$ on FLO)
and generalized ($+2.1\%$ on AWA$2$, $+2.2\%$ on CUB) zero-shot recognition.

提出了一种称为 joint embeddings for zero-shot learning 的方法，通过计算代理任务上的两流网络中的联合图像和文本模型来学习语义基础和丰富的视觉信息，并利用辅助字幕来提高图像和文本表示之间的对齐，从而在多个基准数据集上评估了该方法，在标准的（aPY 上 + 1.6％，FLO 上 + 2.6％）和常规的（AWA2 上 + 2.1％，CUB 上 + 2.2％）零 - shot 识别中提高了现有最先进方法的性能。