In robotic vision, a de-facto paradigm is to learn in simulated environments
and then transfer to real-world applications, which poses an essential
challenge in bridging the sim-to-real domain gap. While mainstream works tackle
this problem in the RGB domain, we focus on depth data synthesis and develop a
range-aware RGB-D data simulation pipeline (RaSim). In particular,
high-fidelity depth data is generated by imitating the imaging principle of
real-world sensors. A range-aware rendering strategy is further introduced to
enrich data diversity. Extensive experiments show that models trained with
RaSim can be directly applied to real-world scenarios without any finetuning
and excel at downstream RGB-D perception tasks.

机器人视觉中的一个主要问题是模拟环境中的学习如何应用到真实世界中，本文的研究关注于深度数据合成，并开发了一种考虑距离的 RGB-D 数据模拟管道（RaSim）。通过模拟真实世界传感器的成像原理来生成高保真度的深度数据，并引入了一个考虑距离的渲染策略来丰富数据多样性。大量实验证明，使用 RaSim 训练的模型可以直接应用于真实世界场景而无需微调，并在后续的 RGB-D 感知任务中表现出色。

RaSim: 用于真实世界应用的基于范围感知的高保真 RGB-D 数据模拟流水线

RaSim: A Range-aware High-fidelity RGB-D Data Simulation Pipeline for  Real-world Applications

Data augmentations are useful in closing the sim-to-real domain gap when
training on synthetic data. This is because they widen the training data
distribution, thus encouraging the model to generalize better to other domains.
Many image augmentation techniques exist, parametrized by different settings,
such as strength and probability. This leads to a large space of different
possible augmentation policies. Some policies work better than others for
overcoming the sim-to-real gap for specific datasets, and it is unclear why.
This paper presents two different interpretable metrics that can be combined to
predict how well a certain augmentation policy will work for a specific
sim-to-real setting, focusing on object detection. We validate our metrics by
training many models with different augmentation policies and showing a strong
correlation with performance on real data. Additionally, we introduce
GeneticAugment, a genetic programming method that can leverage these metrics to
automatically design an augmentation policy for a specific dataset without
needing to train a model.

使用可解释的度量方法，结合遗传编程技术，预测适用于特定 sim-to-real 场景的数据增强策略，并在无需训练模型的情况下为特定数据集设计数据增强策略。

基因学习用于设计模拟到真实数据增强

Genetic Learning for Designing Sim-to-Real Data Augmentations

In industrial manufacturing, numerous tasks of visually inspecting or
detecting specific objects exist that are currently performed manually or by
classical image processing methods. Therefore, introducing recent deep learning
models to industrial environments holds the potential to increase productivity
and enable new applications. However, gathering and labeling sufficient data is
often intractable, complicating the implementation of such projects. Hence,
image synthesis methods are commonly used to generate synthetic training data
from 3D models and annotate them automatically, although it results in a
sim-to-real domain gap. In this paper, we investigate the sim-to-real
generalization performance of standard object detectors on the complex
industrial application of terminal strip object detection. Combining domain
randomization and domain knowledge, we created an image synthesis pipeline for
automatically generating the training data. Moreover, we manually annotated 300
real images of terminal strips for the evaluation. The results show the
cruciality of the objects of interest to have the same scale in either domain.
Nevertheless, under optimized scaling conditions, the sim-to-real performance
difference in mean average precision amounts to 2.69 % for RetinaNet and 0.98 %
for Faster R-CNN, qualifying this approach for industrial requirements.

通过结合领域随机化和领域知识，我们创建了一种图像合成流程，用于自动生成训练数据，并在工业应用中对端子条对象检测的综合泛化性能进行研究。结果表明，在经过优化的尺度条件下，RetinaNet 的模型的模拟到现实平均精度的性能差异为 2.69％，而 Faster R-CNN 的性能差异为 0.98％，从而验证了此方法符合工业要求。