Advances in endoscopy use in surgeries face challenges like inadequate
lighting. Deep learning, notably the Denoising Diffusion Probabilistic Model
(DDPM), holds promise for low-light image enhancement in the medical field.
However, DDPMs are computationally demanding and slow, limiting their practical
medical applications. To bridge this gap, we propose a lightweight DDPM, dubbed
LighTDiff. It adopts a T-shape model architecture to capture global structural
information using low-resolution images and gradually recover the details in
subsequent denoising steps. We further prone the model to significantly reduce
the model size while retaining performance. While discarding certain
downsampling operations to save parameters leads to instability and low
efficiency in convergence during the training, we introduce a Temporal Light
Unit (TLU), a plug-and-play module, for more stable training and better
performance. TLU associates time steps with denoised image features,
establishing temporal dependencies of the denoising steps and improving
denoising outcomes. Moreover, while recovering images using the diffusion
model, potential spectral shifts were noted. We further introduce a Chroma
Balancer (CB) to mitigate this issue. Our LighTDiff outperforms many
competitive LLIE methods with exceptional computational efficiency.

通过使用 T 形模型架构对低分辨率图像进行全局结构信息的捕捉，并逐步恢复细节，我们提出了一种轻量级的 DDPM（LighTDiff），该模型在保留性能的同时显著减小了模型大小。同时，我们引入了时间光单元（TLU）来实现稳定训练和改善去噪结果，通过与去噪图像特征建立时间步骤的关联，建立去噪步骤的时间依赖性。此外，我们还引入了色度平衡器（CB）来缓解扩散模型中可能存在的光谱变化问题。我们的 LighTDiff 在计算效率方面优于许多竞争的 LLIE 方法。

LighTDiff：使用 T-Diffusion 进行手术内窥镜图像低光增强

LighTDiff: Surgical Endoscopic Image Low-Light Enhancement with  T-Diffusion

Vision-centric perception systems for autonomous driving have gained
considerable attention recently due to their cost-effectiveness and
scalability, especially compared to LiDAR-based systems. However, these systems
often struggle in low-light conditions, potentially compromising their
performance and safety. To address this, our paper introduces LightDiff, a
domain-tailored framework designed to enhance the low-light image quality for
autonomous driving applications. Specifically, we employ a multi-condition
controlled diffusion model. LightDiff works without any human-collected paired
data, leveraging a dynamic data degradation process instead. It incorporates a
novel multi-condition adapter that adaptively controls the input weights from
different modalities, including depth maps, RGB images, and text captions, to
effectively illuminate dark scenes while maintaining context consistency.
Furthermore, to align the enhanced images with the detection model's knowledge,
LightDiff employs perception-specific scores as rewards to guide the diffusion
training process through reinforcement learning. Extensive experiments on the
nuScenes datasets demonstrate that LightDiff can significantly improve the
performance of several state-of-the-art 3D detectors in night-time conditions
while achieving high visual quality scores, highlighting its potential to
safeguard autonomous driving.

我们的研究介绍了一种名为 LightDiff 的领域定制框架，该框架旨在增强自动驾驶应用中低光照图像的质量，通过使用多条件控制扩散模型并运用增强学习来引导扩散训练过程，从而在夜间条件下显著提高多个先进的三维检测器的性能并实现高视觉质量分数。