In this paper, we present a novel robust framework for low-level vision tasks, including denoising, object removal, frame interpolation, and super-resolution, that does not require any external training data corpus. Our proposed approach directly learns the weights of neural modules by optimizing over the corrupted test sequence, leveraging the spatio-temporal coherence and internal statistics of videos. Furthermore, we introduce a novel spatial pyramid loss that leverages the property of spatio-temporal patch recurrence in a video across the different scales of the video. This loss enhances robustness to unstructured noise in both the spatial and temporal domains. This further results in our framework being highly robust to degradation in input frames and yields state-of-the-art results on downstream tasks such as denoising, object removal, and frame interpolation. To validate the effectiveness of our approach, we conduct qualitative and quantitative evaluations on standard video datasets such as DAVIS, UCF-101, and VIMEO90K-T.

我们提出了一个新颖的鲁棒性框架，用于低级视觉任务，包括去噪、物体移除、帧插值和超分辨率，不需要任何外部训练数据，通过优化损坏的测试序列来直接学习神经模块的权重，利用视频的时空相干性和内部统计信息。通过引入一种利用视频在不同尺度上的空间-时间块重复性属性的新型空间金字塔损失函数，增强了对空间和时间域中非结构化噪声的鲁棒性。进而，我们的框架在输入帧降质方面具有高鲁棒性，并在去噪、物体移除和帧插值等后续任务上取得了最新成果。为了验证我们方法的有效性，我们在标准视频数据集如DAVIS、UCF-101和VIMEO90K-T上进行了定性和定量评估。

视频动态先验：鲁棒视频增强的内部学习方法