Unsupervised image-to-image translation is the task of translating an image
from one domain to another in the absence of any paired training examples and
tends to be more applicable to practical applications. Nevertheless, the
extension of such synthesis from image-to-image to video-to-video is not
trivial especially when capturing spatio-temporal structures in videos. The
difficulty originates from the aspect that not only the visual appearance in
each frame but also motion between consecutive frames should be realistic and
consistent across transformation. This motivates us to explore both appearance
structure and temporal continuity in video synthesis. In this paper, we present
a new Motion-guided Cycle GAN, dubbed as Mocycle-GAN, that novelly integrates
motion estimation into unpaired video translator. Technically, Mocycle-GAN
capitalizes on three types of constrains: adversarial constraint discriminating
between synthetic and real frame, cycle consistency encouraging an inverse
translation on both frame and motion, and motion translation validating the
transfer of motion between consecutive frames. Extensive experiments are
conducted on video-to-labels and labels-to-video translation, and superior
results are reported when comparing to state-of-the-art methods. More
remarkably, we qualitatively demonstrate our Mocycle-GAN for both
flower-to-flower and ambient condition transfer.

本文提出了一个新的基于运动引导的循环一致性生成对抗网络，称为 Mocycle-GAN，该网络将运动估计集成到无配对视频翻译器中，可用于视频合成。通过对视频到标签和标签到视频的翻译进行广泛的实验，与最先进的方法相比，我们报告了卓越的结果。更明显地，我们定性地演示了我们的 Mocycle-GAN 在花朵和环境条件转移中的应用。