This paper presents a practical application of Relative Trajectory Balance (RTB), a recently introduced off-policy reinforcement learning (RL) objective that can asymptotically solve Bayesian inverse problems optimally. We extend the original work by using RTB to train conditional diffusion model posteriors from pretrained unconditional priors for challenging linear and non-linear inverse problems in vision, and science. We use the objective alongside techniques such as off-policy backtracking exploration to improve training. Importantly, our results show that existing training-free diffusion posterior methods struggle to perform effective posterior inference in latent space due to inherent biases.

本文解决了贝叶斯逆问题中的一个关键挑战，通过引入相对轨迹平衡（RTB）这一离线强化学习目标，优化了逆问题的求解过程。研究展示了利用RTB训练条件扩散模型后验的创新方法，显著提高了线性和非线性逆问题的处理能力，尤其在视觉和科学领域的应用效果显著。

利用扩散先验和离线强化学习解决贝叶斯逆问题