The Composed Image Retrieval (CIR) task aims to retrieve target images using
a composed query consisting of a reference image and a modified text. Advanced
methods often utilize contrastive learning as the optimization objective, which
benefits from adequate positive and negative examples. However, the triplet for
CIR incurs high manual annotation costs, resulting in limited positive
examples. Furthermore, existing methods commonly use in-batch negative
sampling, which reduces the negative number available for the model. To address
the problem of lack of positives, we propose a data generation method by
leveraging a multi-modal large language model to construct triplets for CIR. To
introduce more negatives during fine-tuning, we design a two-stage fine-tuning
framework for CIR, whose second stage introduces plenty of static
representations of negatives to optimize the representation space rapidly. The
above two improvements can be effectively stacked and designed to be
plug-and-play, easily applied to existing CIR models without changing their
original architectures. Extensive experiments and ablation analysis demonstrate
that our method effectively scales positives and negatives and achieves
state-of-the-art results on both FashionIQ and CIRR datasets. In addition, our
methods also perform well in zero-shot composed image retrieval, providing a
new CIR solution for the low-resources scenario.

通过构建三元组并利用大型多模态语言模型，提出了解决复合图像检索问题的数据生成方法，并设计了用于优化表示空间的两阶段微调框架，实验结果显示方法在多个数据集上实现了最先进的结果，并在低资源场景下的复合图像检索中表现出色。

通过缩放的正负样本对比学习提升组合图像检索

Improving Composed Image Retrieval via Contrastive Learning with Scaling  Positives and Negatives

Knowledge graph embedding~(KGE) aims to represent entities and relations into
low-dimensional vectors for many real-world applications. The representations
of entities and relations are learned via contrasting the positive and negative
triplets. Thus, high-quality negative samples are extremely important in KGE.
However, the present KGE models either rely on simple negative sampling
methods, which makes it difficult to obtain informative negative triplets; or
employ complex adversarial methods, which requires more training data and
strategies. In addition, these methods can only construct negative triplets
using the existing entities, which limits the potential to explore harder
negative triplets. To address these issues, we adopt mixing operation in
generating harder negative samples for knowledge graphs and introduce an
inexpensive but effective method called MixKG. Technically, MixKG first
proposes two kinds of criteria to filter hard negative triplets among the
sampled negatives: based on scoring function and based on correct entity
similarity. Then, MixKG synthesizes harder negative samples via the convex
combinations of the paired selected hard negatives. Experiments on two public
datasets and four classical KGE methods show MixKG is superior to previous
negative sampling algorithms.

本文提出了一种基于混合运算的负样本采样方法 MixKG，通过过滤出符合充分性和正确实体相似性两个标准的异常三元组，使用这些三元组生成更高质量的负采样，实验证明该方法优于其他采样算法。