Composed Image Retrieval (CoIR) has recently gained popularity as a task that
considers both text and image queries together, to search for relevant images
in a database. Most CoIR approaches require manually annotated datasets,
comprising image-text-image triplets, where the text describes a modification
from the query image to the target image. However, manual curation of CoIR
triplets is expensive and prevents scalability. In this work, we instead
propose a scalable automatic dataset creation methodology that generates
triplets given video-caption pairs, while also expanding the scope of the task
to include composed video retrieval (CoVR). To this end, we mine paired videos
with a similar caption from a large database, and leverage a large language
model to generate the corresponding modification text. Applying this
methodology to the extensive WebVid2M collection, we automatically construct
our WebVid-CoVR dataset, resulting in 1.6 million triplets. Moreover, we
introduce a new benchmark for CoVR with a manually annotated evaluation set,
along with baseline results. Our experiments further demonstrate that training
a CoVR model on our dataset effectively transfers to CoIR, leading to improved
state-of-the-art performance in the zero-shot setup on both the CIRR and
FashionIQ benchmarks. Our code, datasets, and models are publicly available at
this https URL

通过自动数据集创建方法，我们提出了一个可扩展的 Composed Image Retrieval（CoIR）任务，使用由视频标题对生成的三元组，扩展任务范围到 Composed Video Retrieval（CoVR）。通过在庞大的数据库中挖掘具有相似标题的配对视频，并利用大型语言模型生成相应的修改文本，我们构建了 WebVid-CoVR 数据集，其中包含 1.6 百万个三元组。实验证明，在我们的数据集上训练 CoVR 模型可以有效迁移到 CoIR，提高了 CIRR 和 FashionIQ 基准测试中的最新性能。