Test-time adaptation (TTA) addresses the unforeseen distribution shifts
occurring during test time. In TTA, both performance and, memory and time
consumption serve as crucial considerations. A recent diffusion-based TTA
approach for restoring corrupted images involves image-level updates. However,
using pixel space diffusion significantly increases resource requirements
compared to conventional model updating TTA approaches, revealing limitations
as a TTA method. To address this, we propose a novel TTA method by leveraging a
latent diffusion model (LDM) based image editing model and fine-tuning it with
our newly introduced corruption modeling scheme. This scheme enhances the
robustness of the diffusion model against distribution shifts by creating
(clean, corrupted) image pairs and fine-tuning the model to edit corrupted
images into clean ones. Moreover, we introduce a distilled variant to
accelerate the model for corruption editing using only 4 network function
evaluations (NFEs). We extensively validated our method across various
architectures and datasets including image and video domains. Our model
achieves the best performance with a 100 times faster runtime than that of a
diffusion-based baseline. Furthermore, it outpaces the speed of the model
updating TTA method based on data augmentation threefold, rendering an
image-level updating approach more practical.

利用潜在扩散模型和新引入的损坏建模方案对传统的基于像素扩散的图像修复方案进行改进，从而提高图像修复的鲁棒性和实际性，加快运行速度，并在各种架构和数据集上得到了广泛验证。