Despite recent advances in both model architectures and data augmentation,
multimodal object detectors still barely outperform their LiDAR-only
counterparts. This shortcoming has been attributed to a lack of sufficiently
powerful multimodal data augmentation. To address this, we present SurfaceAug,
a novel ground truth sampling algorithm. SurfaceAug pastes objects by
resampling both images and point clouds, enabling object-level transformations
in both modalities. We evaluate our algorithm by training a multimodal detector
on KITTI and compare its performance to previous works. We show experimentally
that SurfaceAug outperforms existing methods on car detection tasks and
establishes a new state of the art for multimodal ground truth sampling.

通过 SurfaceAug，我们提出了一种新颖的地面真值采样算法，在多模态目标检测中实现了新的性能突破，超越了现有方法，为多模态数据增强提供了有力支持。

SurfaceAug: 闭合多模式基准采样差距

SurfaceAug: Closing the Gap in Multimodal Ground Truth Sampling

Image captioning, an important vision-language task, often requires a
tremendous number of finely labeled image-caption pairs for learning the
underlying alignment between images and texts. In this paper, we proposed a
multimodal data augmentation method, leveraging a recent text-to-image model
called Stable Diffusion, to expand the training set via high-quality generation
of image-caption pairs. Extensive experiments on the MS COCO dataset
demonstrate the advantages of our approach over several benchmark methods, and
particularly a significant boost when having fewer training instances. In
addition, models trained on our augmented datasets also outperform prior
unpaired image captioning methods by a large margin. Finally, further
improvement regarding the training efficiency and effectiveness can be obtained
after intentionally filtering the generated data based on quality assessment.

本研究提出了一种基于多模态数据增强技术的图像字幕生成方法，旨在解决图像字幕对齐困难的问题。实验证明，本方法可以通过高质量生成图像 - 字幕对来扩充训练数据集，从而提高模型的训练效率和预测准确性。

应用扩散模型进行图像字幕的多模态数据增强

Multimodal Data Augmentation for Image Captioning using Diffusion Models

Every hour, huge amounts of visual contents are posted on social media and
user-generated content platforms. To find relevant videos by means of a natural
language query, text-video retrieval methods have received increased attention
over the past few years. Data augmentation techniques were introduced to
increase the performance on unseen test examples by creating new training
samples with the application of semantics-preserving techniques, such as color
space or geometric transformations on images. Yet, these techniques are usually
applied on raw data, leading to more resource-demanding solutions and also
requiring the shareability of the raw data, which may not always be true, e.g.
copyright issues with clips from movies or TV series. To address this
shortcoming, we propose a multimodal data augmentation technique which works in
the feature space and creates new videos and captions by mixing semantically
similar samples. We experiment our solution on a large scale public dataset,
EPIC-Kitchens-100, and achieve considerable improvements over a baseline
method, improved state-of-the-art performance, while at the same time
performing multiple ablation studies. We release code and pretrained models on
Github at this https URL

本文介绍了利用文本 - 视频检索方法，并结合数据增强技术及多模态数据的方法，对大规模公共数据集 EPIC-Kitchens-100 的测试性能进行提升，灵敏的处理方式能以自然语言查询进行相关视频的查找。