Real-world vision tasks frequently suffer from the appearance of unexpected
adverse weather conditions, including rain, haze, snow, and raindrops. In the
last decade, convolutional neural networks and vision transformers have yielded
outstanding results in single-weather video removal. However, due to the
absence of appropriate adaptation, most of them fail to generalize to other
weather conditions. Although ViWS-Net is proposed to remove adverse weather
conditions in videos with a single set of pre-trained weights, it is seriously
blinded by seen weather at train-time and degenerates when coming to unseen
weather during test-time. In this work, we introduce test-time adaptation into
adverse weather removal in videos, and propose the first framework that
integrates test-time adaptation into the iterative diffusion reverse process.
Specifically, we devise a diffusion-based network with a novel temporal noise
model to efficiently explore frame-correlated information in degraded video
clips at training stage. During inference stage, we introduce a proxy task
named Diffusion Tubelet Self-Calibration to learn the primer distribution of
test video stream and optimize the model by approximating the temporal noise
model for online adaptation. Experimental results, on benchmark datasets,
demonstrate that our Test-Time Adaptation method with Diffusion-based
network(Diff-TTA) outperforms state-of-the-art methods in terms of restoring
videos degraded by seen weather conditions. Its generalizable capability is
also validated with unseen weather conditions in both synthesized and
real-world videos.

我们在视频中引入了测试时适应性，并提出了首个将测试时适应性整合到迭代扩散反转过程中的框架，在训练阶段，我们设计了一个基于扩散的网络和一种新颖的时间噪声模型，以高效地探索退化视频剪辑中的帧相关信息，在推理阶段，我们引入了一个名为扩散管道自校准的代理任务，学习测试视频流的引物分布，并通过近似时间噪声模型进行在线适应，实验结果表明，我们的测试时适应性方法与基于扩散的网络（Diff-TTA）在恢复受到已知天气条件影响的视频方面优于最先进的方法，其在合成和实际世界视频中对未知天气条件的普适能力也得到了验证。