Applying an image processing algorithm independently to each video frame
often leads to temporal inconsistency in the resulting video. To address this
issue, we present a novel and general approach for blind video temporal
consistency. Our method is only trained on a pair of original and processed
videos directly instead of a large dataset. Unlike most previous methods that
enforce temporal consistency with optical flow, we show that temporal
consistency can be achieved by training a convolutional neural network on a
video with Deep Video Prior (DVP). Moreover, a carefully designed iteratively
reweighted training strategy is proposed to address the challenging multimodal
inconsistency problem. We demonstrate the effectiveness of our approach on 7
computer vision tasks on videos. Extensive quantitative and perceptual
experiments show that our approach obtains superior performance than
state-of-the-art methods on blind video temporal consistency. We further extend
DVP to video propagation and demonstrate its effectiveness in propagating three
different types of information (color, artistic style, and object
segmentation). A progressive propagation strategy with pseudo labels is also
proposed to enhance DVP's performance on video propagation. Our source codes
are publicly available at this https URL.

本文提出了一种用于盲视频时间一致性的方法，该方法旨在解决仅在每个视频帧上独立应用图像处理算法导致的时间不一致问题。我们展示了通过在 Deep Video Prior（DVP）视频上训练卷积神经网络实现时间一致性的方法，并针对挑战性的多模态不一致性问题提出了一种经过精心设计的迭代加权训练策略。我们通过 7 个计算机视觉任务的广泛定量和感知实验展示了我们的方法的有效性，并证明了我们的方法在盲视频时间一致性方面优于现有技术水平。

视频深度先验及其在视频一致性和传播中的应用

Deep Video Prior for Video Consistency and Propagation

The modern supervised approaches for human image relighting rely on training
data generated from 3D human models. However, such datasets are often small
(e.g., Light Stage data with a small number of individuals) or limited to
diffuse materials (e.g., commercial 3D scanned human models). Thus, the human
relighting techniques suffer from the poor generalization capability and
synthetic-to-real domain gap. In this paper, we propose a two-stage method for
single-image human relighting with domain adaptation. In the first stage, we
train a neural network for diffuse-only relighting. In the second stage, we
train another network for enhancing non-diffuse reflection by learning
residuals between real photos and images reconstructed by the diffuse-only
network. Thanks to the second stage, we can achieve higher generalization
capability against various cloth textures, while reducing the domain gap.
Furthermore, to handle input videos, we integrate illumination-aware deep video
prior to greatly reduce flickering artifacts even with challenging settings
under dynamic illuminations.

本文提出了一种两阶段的人体图像重照明方法，通过神经网络训练和深度视频先验，解决了人体图像重照明的泛化和合成到真实的领域差异问题。

野外人体再照明：领域自适应单目全身人体再照明

Relighting Humans in the Wild: Monocular Full-Body Human Relighting with  Domain Adaptation

Applying image processing algorithms independently to each video frame often
leads to temporal inconsistency in the resulting video. To address this issue,
we present a novel and general approach for blind video temporal consistency.
Our method is only trained on a pair of original and processed videos directly
instead of a large dataset. Unlike most previous methods that enforce temporal
consistency with optical flow, we show that temporal consistency can be
achieved by training a convolutional network on a video with the Deep Video
Prior. Moreover, a carefully designed iteratively reweighted training strategy
is proposed to address the challenging multimodal inconsistency problem. We
demonstrate the effectiveness of our approach on 7 computer vision tasks on
videos. Extensive quantitative and perceptual experiments show that our
approach obtains superior performance than state-of-the-art methods on blind
video temporal consistency. Our source codes are publicly available at
github.com/ChenyangLEI/deep-video-prior.

采用卷积神经网络和 Deep Video Prior 相结合的方法，针对视频中存在的时域不一致问题提出了一个新颖的和普遍的解决方案，并在七项计算机视觉任务中证明了该方法的有效性和性能优于现有技术。