Though dataset distillation has witnessed rapid development in recent years,
the distillation of multimodal data, e.g., image-text pairs, poses unique and
under-explored challenges. Unlike unimodal data, image-text contrastive
learning (ITC) data lack inherent categorization and should instead place
greater emphasis on modality correspondence. In this work, we propose Low-Rank
Similarity Mining (LoRS) for multimodal dataset distillation, that concurrently
distills a ground truth similarity matrix with image-text pairs, and leverages
low-rank factorization for efficiency and scalability. The proposed approach
brings significant improvement to the existing algorithms, marking a
significant contribution to the field of visual-language dataset distillation.
We advocate adopting LoRS as a foundational synthetic data setup for image-text
dataset distillation. Our code is available at
this https URL

提出了一种用于多模态数据集精简的低秩相似度挖掘（LoRS）方法，通过同时挖掘图像 - 文本对中的准确相似度矩阵，并利用低秩因式分解实现高效和可伸缩性，在现有算法上取得了显著的改进，对于视觉语言数据集精简领域做出了重要贡献。

多模态数据集精炼的低秩相似性挖掘

Low-Rank Similarity Mining for Multimodal Dataset Distillation

Employing a dictionary can efficiently rectify the deviation between the
visual prediction and the ground truth in scene text recognition methods.
However, the independence of the dictionary on the visual features may lead to
incorrect rectification of accurate visual predictions. In this paper, we
propose a new dictionary language model leveraging the Scene Image-Text
Matching(SITM) network, which avoids the drawbacks of the explicit dictionary
language model: 1) the independence of the visual features; 2) noisy choice in
candidates etc. The SITM network accomplishes this by using Image-Text
Contrastive (ITC) Learning to match an image with its corresponding text among
candidates in the inference stage. ITC is widely used in vision-language
learning to pull the positive image-text pair closer in feature space. Inspired
by ITC, the SITM network combines the visual features and the text features of
all candidates to identify the candidate with the minimum distance in the
feature space. Our lexicon method achieves better results(93.8\% accuracy) than
the ordinary method results(92.1\% accuracy) on six mainstream benchmarks.
Additionally, we integrate our method with ABINet and establish new
state-of-the-art results on several benchmarks.

本文提出了一种基于场景图像 - 文本匹配（SITM）网络的新字典语言模型，该模型通过使用图像 - 文本对比（ITC）学习在特征空间中将正面的图像 - 文本对更紧密地匹配，从而避免了显式字典语言模型中的缺点。在六个主流基准测试中，我们的词库方法取得了更好的结果（93.8％准确性），并且与 ABINet 集成后在几个基准测试中建立了新的最优结果。