In this study, we present an efficient and effective approach for achieving
temporally consistent synthetic-to-real video translation in videos of varying
lengths. Our method leverages off-the-shelf conditional image diffusion models,
allowing us to perform multiple synthetic-to-real image generations in
parallel. By utilizing the available optical flow information from the
synthetic videos, our approach seamlessly enforces temporal consistency among
corresponding pixels across frames. This is achieved through joint noise
optimization, effectively minimizing spatial and temporal discrepancies. To the
best of our knowledge, our proposed method is the first to accomplish diverse
and temporally consistent synthetic-to-real video translation using conditional
image diffusion models. Furthermore, our approach does not require any training
or fine-tuning of the diffusion models. Extensive experiments conducted on
various benchmarks for synthetic-to-real video translation demonstrate the
effectiveness of our approach, both quantitatively and qualitatively. Finally,
we show that our method outperforms other baseline methods in terms of both
temporal consistency and visual quality.

本研究提出了一种高效有效的方法，通过利用条件图像扩散模型实现长度可变视频中时间一致的合成到真实视频转换，同时保持视频的时空一致性。我们通过使用可用的合成视频的光流信息，通过联合噪声优化有效地最小化了时空不一致性，实现对多个合成图像生成的平行化。通过在各种综合基准上进行的大量实验证明了我们的方法的有效性，并且我们的方法不需要对扩散模型进行任何培训或微调。 最后，我们证明了我们的方法在时空一致性和视觉质量方面均优于其他基线方法。