Video understanding calls for a model to learn the characteristic interplay
between static scene content and its dynamics: Given an image, the model must
be able to predict a future progression of the portrayed scene and, conversely,
a video should be explained in terms of its static image content and all the
remaining characteristics not present in the initial frame. This naturally
suggests a bijective mapping between the video domain and the static content as
well as residual information. In contrast to common stochastic image-to-video
synthesis, such a model does not merely generate arbitrary videos progressing
the initial image. Given this image, it rather provides a one-to-one mapping
between the residual vectors and the video with stochastic outcomes when
sampling. The approach is naturally implemented using a conditional invertible
neural network (cINN) that can explain videos by independently modelling static
and other video characteristics, thus laying the basis for controlled video
synthesis. Experiments on four diverse video datasets demonstrate the
effectiveness of our approach in terms of both the quality and diversity of the
synthesized results. Our project page is available at this https URL

通过使用条件可逆神经网络来独立地建模静态内容和其他视频特征，从而为受控视频合成奠定基础。在四个不同的视频数据集上进行的实验表明，该方法在合成结果的质量和多样性方面都具有很好的效果。

使用 cINNs 的随机图像到视频合成

Stochastic Image-to-Video Synthesis using cINNs

We introduce a new architecture called a conditional invertible neural
network (cINN), and use it to address the task of diverse image-to-image
translation for natural images. This is not easily possible with existing INN
models due to some fundamental limitations. The cINN combines the purely
generative INN model with an unconstrained feed-forward network, which
efficiently preprocesses the conditioning image into maximally informative
features. All parameters of a cINN are jointly optimized with a stable, maximum
likelihood-based training procedure. Even though INN-based models have received
far less attention in the literature than GANs, they have been shown to have
some remarkable properties absent in GANs, e.g. apparent immunity to mode
collapse. We find that our cINNs leverage these properties for image-to-image
translation, demonstrated on day to night translation and image colorization.
Furthermore, we take advantage of our bidirectional cINN architecture to
explore and manipulate emergent properties of the latent space, such as
changing the image style in an intuitive way.

介绍了一种新的结构，称为条件可逆神经网络（cINN），并将其用于解决自然图像的多样化图像到图像的转化任务。cINN 结合了纯生成 INN 模型和一个无限制的前馈网络，有效地将调节图像预处理为最大信息特征。通过最大似然优化，稳定地联合优化 cINN 的所有参数。利用双向 cINN 结构，进一步探索和操作潜在空间的新兴属性，例如以直观的方式更改图像风格。

基于条件逆神经网络的多样化图像到图像转换

Conditional Invertible Neural Networks for Diverse Image-to-Image  Translation

In this work, we address the task of natural image generation guided by a
conditioning input. We introduce a new architecture called conditional
invertible neural network (cINN). The cINN combines the purely generative INN
model with an unconstrained feed-forward network, which efficiently
preprocesses the conditioning input into useful features. All parameters of the
cINN are jointly optimized with a stable, maximum likelihood-based training
procedure. By construction, the cINN does not experience mode collapse and
generates diverse samples, in contrast to e.g. cGANs. At the same time our
model produces sharp images since no reconstruction loss is required, in
contrast to e.g. VAEs. We demonstrate these properties for the tasks of MNIST
digit generation and image colorization. Furthermore, we take advantage of our
bi-directional cINN architecture to explore and manipulate emergent properties
of the latent space, such as changing the image style in an intuitive way.

通过使用一种新的网络模型，即条件可逆神经网络，我们可以对特定条件下的自然图像生成进行有效的前处理，以便于训练出不会出现模式崩溃并生成多样化图像的模型。同时，我们利用双向 cINN 架构探索和操作潜在空间的新特性，例如通过直观方式改变图像风格。