Text-image composed retrieval aims to retrieve the target image through the
composed query, which is specified in the form of an image plus some text that
describes desired modifications to the input image. It has recently attracted
attention due to its ability to leverage both information-rich images and
concise language to precisely express the requirements for target images.
However, the robustness of these approaches against real-world corruptions or
further text understanding has never been studied. In this paper, we perform
the first robustness study and establish three new diversified benchmarks for
systematic analysis of text-image composed retrieval against natural
corruptions in both vision and text and further probe textural understanding.
For natural corruption analysis, we introduce two new large-scale benchmark
datasets, CIRR-C and FashionIQ-C for testing in open domain and fashion domain
respectively, both of which apply 15 visual corruptions and 7 textural
corruptions. For textural understanding analysis, we introduce a new diagnostic
dataset CIRR-D by expanding the original raw data with synthetic data, which
contains modified text to better probe textual understanding ability including
numerical variation, attribute variation, object removal, background variation,
and fine-grained evaluation. The code and benchmark datasets are available at
this https URL

通过图文组合检索，通过由图像加上一些描述所构成的查询准确检索目标图像，在真实世界的损坏和进一步的文本理解下进行了坚固性研究，建立了三个用于系统分析的新的多样化基准，对视觉和文本的图文组合检索进行评估，包括自然失真的分析和文本理解的论证。

文本 - 图像组合检索的鲁棒性评估

Benchmarking Robustness of Text-Image Composed Retrieval

Benefiting from large-scale pre-training, we have witnessed significant
performance boost on the popular Visual Question Answering (VQA) task. Despite
rapid progress, it remains unclear whether these state-of-the-art (SOTA) models
are robust when encountering examples in the wild. To study this, we introduce
Adversarial VQA, a new large-scale VQA benchmark, collected iteratively via an
adversarial human-and-model-in-the-loop procedure. Through this new benchmark,
we discover several interesting findings. (i) Surprisingly, we find that during
dataset collection, non-expert annotators can easily attack SOTA VQA models
successfully. (ii) Both large-scale pre-trained models and adversarial training
methods achieve far worse performance on the new benchmark than over standard
VQA v2 dataset, revealing the fragility of these models while demonstrating the
effectiveness of our adversarial dataset. (iii) When used for data
augmentation, our dataset can effectively boost model performance on other
robust VQA benchmarks. We hope our Adversarial VQA dataset can shed new light
on robustness study in the community and serve as a valuable benchmark for
future work.

通过引入 Adversarial VQA 作为一个新的大型 VQA 基准测试，我们发现非专业注释者可以轻松成功攻击 SOTA VQA 模型，而预训练模型和对抗训练方法在新基准测试中效果远不如 VQA v2 基准测试，并且我们的数据集可有效提高模型在其他稳健性 VQA 基准测试中的性能。