Deep learning-based image restoration methods have achieved promising
performance. However, how to faithfully preserve the structure of the original
image remains challenging. To address this challenge, we propose a novel
Residual-Conditioned Optimal Transport (RCOT) approach, which models the image
restoration as an optimal transport (OT) problem for both unpaired and paired
settings, integrating the transport residual as a unique degradation-specific
cue for both the transport cost and the transport map. Specifically, we first
formalize a Fourier residual-guided OT objective by incorporating the
degradation-specific information of the residual into the transport cost. Based
on the dual form of the OT formulation, we design the transport map as a
two-pass RCOT map that comprises a base model and a refinement process, in
which the transport residual is computed by the base model in the first pass
and then encoded as a degradation-specific embedding to condition the
second-pass restoration. By duality, the RCOT problem is transformed into a
minimax optimization problem, which can be solved by adversarially training
neural networks. Extensive experiments on multiple restoration tasks show the
effectiveness of our approach in terms of both distortion measures and
perceptual quality. Particularly, RCOT restores images with more faithful
structural details compared to state-of-the-art methods.

深度学习图像修复中，我们提出了一种新的残余条件的最优传输（RCOT）方法，该方法将图像修复建模为一种最优传输问题，通过整合传输残差作为独特的特定退化线索来保留原始图像的结构细节，并通过对抗训练神经网络来解决。与现有方法相比，RCOT 方法在多个修复任务中显示出更好的失真度量和感知质量，可以更忠实地恢复图像的结构细节。