We present the first publicly available RGB-thermal dataset designed for
aerial robotics operating in natural environments. Our dataset captures a
variety of terrains across the continental United States, including rivers,
lakes, coastlines, deserts, and forests, and consists of synchronized RGB,
long-wave thermal, global positioning, and inertial data. Furthermore, we
provide semantic segmentation annotations for 10 classes commonly encountered
in natural settings in order to facilitate the development of perception
algorithms robust to adverse weather and nighttime conditions. Using this
dataset, we propose new and challenging benchmarks for thermal and RGB-thermal
semantic segmentation, RGB-to-thermal image translation, and visual-inertial
odometry. We present extensive results using state-of-the-art methods and
highlight the challenges posed by temporal and geographical domain shifts in
our data. Dataset and accompanying code will be provided at
this https URL

本研究提供了首个在自然环境中进行航空机器人操作的公开 RGB - 热数据集。该数据集捕捉了美国大陆各种地形，包括河流、湖泊、海岸线、沙漠和森林，并包含了同步的 RGB、长波热、全球定位和惯性数据。此外，我们为自然环境常见的 10 个类别提供语义分割注释，以促进对逆境天气和夜间条件下鲁棒的感知算法的开发。利用该数据集，我们提出了热和 RGB - 热语义分割、RGB 到热图像转换和视觉惯性测程的新的具有挑战性的基准测试。通过使用最先进的方法进行广泛的实验并强调了我们的数据中面临的时间和地理领域变化所带来的挑战。数据集和附带的代码将在此 https 网址上提供。

CART: Caltech 野外空中 RGB - 热成像数据集

CART: Caltech Aerial RGB-Thermal Dataset in the Wild

Inferring past human motion from RGB images is challenging due to the
inherent uncertainty of the prediction problem. Thermal images, on the other
hand, encode traces of past human-object interactions left in the environment
via thermal radiation measurement. Based on this observation, we collect the
first RGB-Thermal dataset for human motion analysis, dubbed Thermal-IM. Then we
develop a three-stage neural network model for accurate past human pose
estimation. Comprehensive experiments show that thermal cues significantly
reduce the ambiguities of this task, and the proposed model achieves remarkable
performance. The dataset is available at
this https URL

该研究开发了一个三阶段的神经网络模型，利用 RGB - 热成像数据集进行过去的人体姿势估计，并表明热信号可显著降低该任务的不确定性。

3 秒钟前发生了什么？热成像推断过去

What Happened 3 Seconds Ago? Inferring the Past with Thermal Imaging

The majority of learning-based semantic segmentation methods are optimized
for daytime scenarios and favorable lighting conditions. Real-world driving
scenarios, however, entail adverse environmental conditions such as nighttime
illumination or glare which remain a challenge for existing approaches. In this
work, we propose a multimodal semantic segmentation model that can be applied
during daytime and nighttime. To this end, besides RGB images, we leverage
thermal images, making our network significantly more robust. We avoid the
expensive annotation of nighttime images by leveraging an existing daytime
RGB-dataset and propose a teacher-student training approach that transfers the
dataset's knowledge to the nighttime domain. We further employ a domain
adaptation method to align the learned feature spaces across the domains and
propose a novel two-stage training scheme. Furthermore, due to a lack of
thermal data for autonomous driving, we present a new dataset comprising over
20,000 time-synchronized and aligned RGB-thermal image pairs. In this context,
we also present a novel target-less calibration method that allows for
automatic robust extrinsic and intrinsic thermal camera calibration. Among
others, we employ our new dataset to show state-of-the-art results for
nighttime semantic segmentation.

本文提出了一种多模态语义分割模型，可以适用于日间和夜间场景，通过引入热成像技术和 RGB 照片，利用现有白天 RGB 数据集实现对夜间图像的学习，并提出了一个新的域对齐训练方法，在自动驾驶方面获得了最新的夜间语义分割结果。