We introduce DeepIR, a new thermal image processing framework that combines
physically accurate sensor modeling with deep network-based image
representation. Our key enabling observations are that the images captured by
thermal sensors can be factored into slowly changing, scene-independent sensor
non-uniformities (that can be accurately modeled using physics) and a
scene-specific radiance flux (that is well-represented using a deep
network-based regularizer). DeepIR requires neither training data nor periodic
ground-truth calibration with a known black body target--making it well suited
for practical computer vision tasks. We demonstrate the power of going DeepIR
by developing new denoising and super-resolution algorithms that exploit
multiple images of the scene captured with camera jitter. Simulated and real
data experiments demonstrate that DeepIR can perform high-quality
non-uniformity correction with as few as three images, achieving a 10dB PSNR
improvement over competing approaches.

DeepIR 是一个新的热成像处理框架，结合了精确的传感器建模和深度网络图像表示，可用于开展无需训练数据和已知黑体目标的计算机视觉任务，并采用多张图像噪声抑制和超分辨率算法，实现高质量的非均匀校正。

基于物理启发的深度网络的热图像处理

Thermal Image Processing via Physics-Inspired Deep Networks

In the autonomous driving domain, data collection and annotation from real
vehicles are expensive and sometimes unsafe. Simulators are often used for data
augmentation, which requires realistic sensor models that are hard to formulate
and model in closed forms. Instead, sensors models can be learned from real
data. The main challenge is the absence of paired data set, which makes
traditional supervised learning techniques not suitable. In this work, we
formulate the problem as image translation from unpaired data and employ
CycleGANs to solve the sensor modeling problem for LiDAR, to produce realistic
LiDAR from simulated LiDAR (sim2real). Further, we generate high-resolution,
realistic LiDAR from lower resolution one (real2real). The LiDAR 3D point cloud
is processed in Bird-eye View and Polar 2D representations. The experimental
results show a high potential of the proposed approach.

通过 CycleGANs 从未配对的数据中进行图像转换，以实现从模拟 LiDAR 到真实 LiDAR（sim2real）的传感器建模，从低分辨率到高分辨率的真实 LiDAR（real2real）的生成以及使用鸟瞰图和极坐标 2D 表示法处理 LiDAR 3D 点云。实验结果表明，该方法具有很高的潜力。