Video-to-Video synthesis (Vid2Vid) has achieved remarkable results in
generating a photo-realistic video from a sequence of semantic maps. However,
this pipeline suffers from high computational cost and long inference latency,
which largely depends on two essential factors: 1) network architecture
parameters, 2) sequential data stream. Recently, the parameters of image-based
generative models have been significantly compressed via more efficient network
architectures. Nevertheless, existing methods mainly focus on slimming network
architectures and ignore the size of the sequential data stream. Moreover, due
to the lack of temporal coherence, image-based compression is not sufficient
for the compression of the video task. In this paper, we present a
spatial-temporal compression framework, \textbf{Fast-Vid2Vid}, which focuses on
data aspects of generative models. It makes the first attempt at time dimension
to reduce computational resources and accelerate inference. Specifically, we
compress the input data stream spatially and reduce the temporal redundancy.
After the proposed spatial-temporal knowledge distillation, our model can
synthesize key-frames using the low-resolution data stream. Finally,
Fast-Vid2Vid interpolates intermediate frames by motion compensation with
slight latency. On standard benchmarks, Fast-Vid2Vid achieves around real-time
performance as 20 FPS and saves around 8x computational cost on a single V100
GPU.

本文提出了一种名为 Fast-Vid2Vid 的新型空间时间压缩框架，旨在减少计算资源并加速推理，通过对生成模型的数据方面进行压缩处理，使模型实现低分辨率数据流合成 / 高效中间帧插值，并在标准基准测试中实现约 20 FPS 的实时性能，同时在单个 V100 GPU 上节约约 8 倍的计算成本。