This paper describes our zero-shot approaches for the Visual Word Sense
Disambiguation (VWSD) Task in English. Our preliminary study shows that the
simple approach of matching candidate images with the phrase using CLIP suffers
from the many-to-many nature of image-text pairs. We find that the CLIP text
encoder may have limited abilities in capturing the compositionality in natural
language. Conversely, the descriptive focus of the phrase varies from instance
to instance. We address these issues in our two systems, Augment-CLIP and
Stable Diffusion Sampling (SD Sampling). Augment-CLIP augments the text prompt
by generating sentences that contain the context phrase with the help of large
language models (LLMs). We further explore CLIP models in other languages, as
the an ambiguous word may be translated into an unambiguous one in the other
language. SD Sampling uses text-to-image Stable Diffusion to generate multiple
images from the given phrase, increasing the likelihood that a subset of images
match the one that paired with the text.

本文有关英文文本下的视觉词义消岐，作者通过两种方法 Augment-CLIP 与 Stable Diffusion Sampling 解决了匹配过程中复杂的组合问题，其中 Augment-CLIP 采用了较大的语言模型来生成文本提示。

SemEval-2023 任务 1 的增强器：通过提示增强和文本图像扩散增强 CLIP 在处理零样本视觉词义消歧中的复合性和歧义性能力

Augmenters at SemEval-2023 Task 1: Enhancing CLIP in Handling  Compositionality and Ambiguity for Zero-Shot Visual WSD through Prompt  Augmentation and Text-To-Image Diffusion