Comparing two images in terms of Commonalities and Differences (CaD) is a
fundamental human capability that forms the basis of advanced visual reasoning
and interpretation. It is essential for the generation of detailed and
contextually relevant descriptions, performing comparative analysis, novelty
detection, and making informed decisions based on visual data. However,
surprisingly, little attention has been given to these fundamental concepts in
the best current mimic of human visual intelligence - Large Multimodal Models
(LMMs). We develop and contribute a new two-phase approach CaD-VI for
collecting synthetic visual instructions, together with an
instruction-following dataset CaD-Inst containing 349K image pairs with CaD
instructions collected using CaD-VI. Our approach significantly improves the
CaD spotting capabilities in LMMs, advancing the SOTA on a diverse set of
related tasks by up to 17.5%. It is also complementary to existing
difference-only instruction datasets, allowing automatic targeted refinement of
those resources increasing their effectiveness for CaD tuning by up to 10%.
Additionally, we propose an evaluation benchmark with 7.5K open-ended QAs to
assess the CaD understanding abilities of LMMs.

比较两个图像的共性和差异（CaD）是一种基本的人类能力，它是高级视觉推理和解释的基础，对于生成详细的和与上下文相关的描述、执行比较分析、新颖性检测以及基于视觉数据做出明智决策至关重要。然而，令人惊讶的是，最先进的人类视觉智能模型 - 大型多模态模型（LMMs）在这些基本概念上付出的关注甚少。我们开发并提出了一种新的两阶段方法 CaD-VI 来收集合成视觉指令，同时还提供了一个包含 349K 图像对的 CaD 指令数据集 CaD-Inst，使用 CaD-VI 收集。我们的方法显著提高了 LMMs 的 CaD 定位能力，通过对一系列相关任务的研究，性能优于目前技术水准 17.5%。它还可以与现有的仅差异指令数据集相互补充，自动针对性地优化这些资源，将它们在 CaD 调整上的有效性提高 10%。此外，我们还提出了一个包含 7.5K 开放式问答的评估基准，以评估 LMMs 的 CaD 理解能力。