Collecting well-matched multimedia datasets is crucial for training
cross-modal retrieval models. However, in real-world scenarios, massive
multimodal data are harvested from the Internet, which inevitably contains
Partially Mismatched Pairs (PMPs). Undoubtedly, such semantical irrelevant data
will remarkably harm the cross-modal retrieval performance. Previous efforts
tend to mitigate this problem by estimating a soft correspondence to
down-weight the contribution of PMPs. In this paper, we aim to address this
challenge from a new perspective: the potential semantic similarity among
unpaired samples makes it possible to excavate useful knowledge from mismatched
pairs. To achieve this, we propose L2RM, a general framework based on Optimal
Transport (OT) that learns to rematch mismatched pairs. In detail, L2RM aims to
generate refined alignments by seeking a minimal-cost transport plan across
different modalities. To formalize the rematching idea in OT, first, we propose
a self-supervised cost function that automatically learns from explicit
similarity-cost mapping relation. Second, we present to model a partial OT
problem while restricting the transport among false positives to further boost
refined alignments. Extensive experiments on three benchmarks demonstrate our
L2RM significantly improves the robustness against PMPs for existing models.
The code is available at this https URL

通过最佳传输方法和重新配对，本研究提出了一种处理来自互联网上收集的异构数据的实用框架，该框架在已有模型的基础上显著提高了鲁棒性。

学习重新匹配不匹配的配对以实现鲁棒的跨模态检索

Learning to Rematch Mismatched Pairs for Robust Cross-Modal Retrieval

Traditional learning systems are trained in closed-world for a fixed number
of classes, and need pre-collected datasets in advance. However, new classes
often emerge in real-world applications and should be learned incrementally.
For example, in electronic commerce, new types of products appear daily, and in
a social media community, new topics emerge frequently. Under such
circumstances, incremental models should learn several new classes at a time
without forgetting. We find a strong correlation between old and new classes in
incremental learning, which can be applied to relate and facilitate different
learning stages mutually. As a result, we propose CO-transport for class
Incremental Learning (COIL), which learns to relate across incremental tasks
with the class-wise semantic relationship. In detail, co-transport has two
aspects: prospective transport tries to augment the old classifier with optimal
transported knowledge as fast model adaptation. Retrospective transport aims to
transport new class classifiers backward as old ones to overcome forgetting.
With these transports, COIL efficiently adapts to new tasks, and stably resists
forgetting. Experiments on benchmark and real-world multimedia datasets
validate the effectiveness of our proposed method.

CO-transport for Class Incremental Learning (COIL) is proposed as a method for incremental learning that utilizes class-wise semantic relationships and prospective and retrospective transport to adapt to new tasks and resist forgetting, as validated in experiments on benchmark and real-world multimedia datasets.

类增量学习的共同传输

Co-Transport for Class-Incremental Learning

Past research has challenged us with the task of showing relational patterns
between text-based data and then clustering for predictive analysis using Golay
Code technique. We focus on a novel approach to extract metaknowledge in
multimedia datasets. Our collaboration has been an on-going task of studying
the relational patterns between datapoints based on metafeatures extracted from
metaknowledge in multimedia datasets. Those selected are significant to suit
the mining technique we applied, Golay Code algorithm. In this research paper
we summarize findings in optimization of metaknowledge representation for
23-bit representation of structured and unstructured multimedia data in order
to

本研究通过提取多媒体数据集中的元特征，针对高纬结构化和非结构化数据的 23 位表示进行元知识表征的优化，以适用于 Golay 码算法的挖掘技术，研究数据点之间的关系模式和聚类预测分析方法。