Existing unsupervised video-to-video translation methods fail to produce
translated videos which are frame-wise realistic, semantic information
preserving and video-level consistent. In this work, we propose UVIT, a novel
unsupervised video-to-video translation model. Our model decomposes the style
and the content, uses the specialized encoder-decoder structure and propagates
the inter-frame information through bidirectional recurrent neural network
(RNN) units. The style-content decomposition mechanism enables us to achieve
style consistent video translation results as well as provides us with a good
interface for modality flexible translation. In addition, by changing the input
frames and style codes incorporated in our translation, we propose a video
interpolation loss, which captures temporal information within the sequence to
train our building blocks in a self-supervised manner. Our model can produce
photo-realistic, spatio-temporal consistent translated videos in a multimodal
way. Subjective and objective experimental results validate the superiority of
our model over existing methods. More details can be found on our project
website: this https URL

该研究提出了一个名为 UVIT 的新型无监督视频到视频翻译模型，采用专用编码器 - 解码器结构和双向循环神经网络单元，通过风格内容分解机制使得其能够实现样式一致的视频翻译结果，并通过更改输入帧和风格码提出一种视频内插损失，可以以自我监督的方式训练建筑块。该模型能够以多模式方式生成现实照片和时空一致性的翻译视频。