Open-vocabulary vision-language models (VLMs) like CLIP, trained using
contrastive loss, have emerged as a promising new paradigm for text-to-image
retrieval. However, do VLMs understand compound nouns (CNs) (e.g., lab coat) as
well as they understand nouns (e.g., lab)? We curate Compun, a novel benchmark
with 400 unique and commonly used CNs, to evaluate the effectiveness of VLMs in
interpreting CNs. The Compun benchmark challenges a VLM for text-to-image
retrieval where, given a text prompt with a CN, the task is to select the
correct image that shows the CN among a pair of distractor images that show the
constituent nouns that make up the CN. Next, we perform an in-depth analysis to
highlight CLIPs' limited understanding of certain types of CNs. Finally, we
present an alternative framework that moves beyond hand-written templates for
text prompts widely used by CLIP-like models. We employ a Large Language Model
to generate multiple diverse captions that include the CN as an object in the
scene described by the caption. Our proposed method improves CN understanding
of CLIP by 8.25% on Compun. Code and benchmark are available at:
this https URL

开放词汇视觉 - 语言模型（VLMs）如 CLIP 是一种有前景的文本到图像检索方法，然而，对于复合名词（CN）是否能像对待名词一样理解得好呢？本研究构建了 Compun 基准测试以评估 VLMs 在解释 CNs 方面的有效性，并深入分析了 CLIP 对某些类型的 CNs 的有限理解。同时，提出了一种超越手写模板的替代框架，通过利用大型语言模型生成包含 CNs 的多样化描述来提高 CLIP 对 CNs 的理解。通过我们的方法在 Compun 上将 CN 的理解能力提高了 8.25%。

视觉 - 语言模型是否理解复合名词？

Do Vision-Language Models Understand Compound Nouns?

We introduce temporally and contextually-aware models for the novel task of
predicting unseen but plausible concepts, as conveyed by noun-noun compounds in
a time-stamped corpus. We train compositional models on observed compounds,
more specifically the composed distributed representations of their
constituents across a time-stamped corpus, while giving it corrupted instances
(where head or modifier are replaced by a random constituent) as negative
evidence. The model captures generalisations over this data and learns what
combinations give rise to plausible compounds and which ones do not. After
training, we query the model for the plausibility of automatically generated
novel combinations and verify whether the classifications are accurate. For our
best model, we find that in around 85% of the cases, the novel compounds
generated are attested in previously unseen data. An additional estimated 5%
are plausible despite not being attested in the recent corpus, based on
judgments from independent human raters.

本文介绍了一种感知时间和语境的模型用于预测未知合成名词所表达的可信概念，该模型在在一个时间戳语料库上训练，可以识别真实的和虚假的合成名词。经测试，该模型在多大约 85% 的情况下能正确分类自动生成的新的合成名词并有五分之一的新合成名词被评定为可信。