Inverse problems arise in a multitude of applications, where the goal is to
recover a clean signal from noisy and possibly (non)linear observations. The
difficulty of a reconstruction problem depends on multiple factors, such as the
structure of the ground truth signal, the severity of the degradation, the
implicit bias of the reconstruction model and the complex interactions between
the above factors. This results in natural sample-by-sample variation in the
difficulty of a reconstruction task, which is often overlooked by contemporary
techniques. Recently, diffusion-based inverse problem solvers have established
new state-of-the-art in various reconstruction tasks. However, they have the
drawback of being computationally prohibitive. Our key observation in this
paper is that most existing solvers lack the ability to adapt their compute
power to the difficulty of the reconstruction task, resulting in long inference
times, subpar performance and wasteful resource allocation. We propose a novel
method that we call severity encoding, to estimate the degradation severity of
noisy, degraded signals in the latent space of an autoencoder. We show that the
estimated severity has strong correlation with the true corruption level and
can give useful hints at the difficulty of reconstruction problems on a
sample-by-sample basis. Furthermore, we propose a reconstruction method based
on latent diffusion models that leverages the predicted degradation severities
to fine-tune the reverse diffusion sampling trajectory and thus achieve
sample-adaptive inference times. We utilize latent diffusion posterior sampling
to maintain data consistency with observations. We perform experiments on both
linear and nonlinear inverse problems and demonstrate that our technique
achieves performance comparable to state-of-the-art diffusion-based techniques,
with significant improvements in computational efficiency.

我们提出了一种名为 “严重性编码” 的新方法，通过在自编码器的潜在空间中估计噪声、退化信号的退化严重程度，从而在样本自适应推断时间、重建问题的难度上取得了显著性进展。同时，利用潜在扩散模型基于预测的退化严重程度来调整反向扩散采样轨迹，从而在维持与观测一致性的同时实现样本自适应的推断时间，与现有扩散法技术相比，我们的技术在计算效率方面有明显提升。