Training multimodal networks requires a vast amount of data due to their
larger parameter space compared to unimodal networks. Active learning is a
widely used technique for reducing data annotation costs by selecting only
those samples that could contribute to improving model performance. However,
current active learning strategies are mostly designed for unimodal tasks, and
when applied to multimodal data, they often result in biased sample selection
from the dominant modality. This unfairness hinders balanced multimodal
learning, which is crucial for achieving optimal performance. To address this
issue, we propose three guidelines for designing a more balanced multimodal
active learning strategy. Following these guidelines, a novel approach is
proposed to achieve more fair data selection by modulating the gradient
embedding with the dominance degree among modalities. Our studies demonstrate
that the proposed method achieves more balanced multimodal learning by avoiding
greedy sample selection from the dominant modality. Our approach outperforms
existing active learning strategies on a variety of multimodal classification
tasks. Overall, our work highlights the importance of balancing sample
selection in multimodal active learning and provides a practical solution for
achieving more balanced active learning for multimodal classification.

本文提出了三个设计均衡的多模式主动学习策略的指导方针，并通过调节模态间的支配度来调节梯度嵌入，从而实现更公平的数据选择，本方法在多种多模态分类任务上表现优异，实现了更平衡的多模态学习。

多模态分类的平衡主动学习

Towards Balanced Active Learning for Multimodal Classification

Instance recognition is rapidly advanced along with the developments of
various deep convolutional neural networks. Compared to the architectures of
networks, the training process, which is also crucial to the success of
detectors, has received relatively less attention. In this work, we carefully
revisit the standard training practice of detectors, and find that the
detection performance is often limited by the imbalance during the training
process, which generally consists in three levels - sample level, feature
level, and objective level. To mitigate the adverse effects caused thereby, we
propose Libra R-CNN, a simple yet effective framework towards balanced learning
for instance recognition. It integrates IoU-balanced sampling, balanced feature
pyramid, and objective re-weighting, respectively for reducing the imbalance at
sample, feature, and objective level. Extensive experiments conducted on MS
COCO, LVIS and Pascal VOC datasets prove the effectiveness of the overall
balanced design.

本文提出了一种针对实例识别的平衡学习框架 Libra R-CNN，其中包括 IoU 平衡采样、平衡特征金字塔和目标重加权，以分别减少样本级别、特征级别和目标级别的不平衡性，通过对 MS COCO、LVIS 和 Pascal VOC 等数据集的广泛实验，证明了整体平衡设计的有效性。

实例识别的平衡学习方法研究

Towards Balanced Learning for Instance Recognition

Compared with model architectures, the training process, which is also
crucial to the success of detectors, has received relatively less attention in
object detection. In this work, we carefully revisit the standard training
practice of detectors, and find that the detection performance is often limited
by the imbalance during the training process, which generally consists in three
levels - sample level, feature level, and objective level. To mitigate the
adverse effects caused thereby, we propose Libra R-CNN, a simple but effective
framework towards balanced learning for object detection. It integrates three
novel components: IoU-balanced sampling, balanced feature pyramid, and balanced
L1 loss, respectively for reducing the imbalance at sample, feature, and
objective level. Benefitted from the overall balanced design, Libra R-CNN
significantly improves the detection performance. Without bells and whistles,
it achieves 2.5 points and 2.0 points higher Average Precision (AP) than FPN
Faster R-CNN and RetinaNet respectively on MSCOCO.

本研究提出了 Libra R-CNN 框架实现平衡的目标检测训练，通过三个组件，在样本、特征和目标级别减少不平衡，显著提高检测性能。