Novel view synthesis of static scenes has achieved remarkable advancements in
producing photo-realistic results. However, key challenges remain for immersive
rendering of dynamic scenes. One of the seminal image-based rendering method,
the multi-plane image (MPI), produces high novel-view synthesis quality for
static scenes. But modelling dynamic contents by MPI is not studied. In this
paper, we propose a novel Temporal-MPI representation which is able to encode
the rich 3D and dynamic variation information throughout the entire video as
compact temporal basis and coefficients jointly learned. Time-instance MPI for
rendering can be generated efficiently using mini-seconds by linear
combinations of temporal basis and coefficients from Temporal-MPI. Thus
novel-views at arbitrary time-instance will be able to be rendered via
Temporal-MPI in real-time with high visual quality. Our method is trained and
evaluated on Nvidia Dynamic Scene Dataset. We show that our proposed Temporal-
MPI is much faster and more compact compared with other state-of-the-art
dynamic scene modelling methods.

本文提出了一种新的基于时间的多平面图像（Temporal-MPI）动态场景表示方法，它能够以紧凑的时间基和系数来编码整个视频的丰富 3D 和动态变化信息，并通过 Temporal-MPI 实时高质量地生成任意时刻的新视角。我们的方法在 Nvidia 动态场景数据集上进行训练和评估，证明了其比其他最先进的动态场景建模方法更快速更紧凑。

Temporal-MPI: 基于时间基础学习的动态场景建模多平面图像实现

Temporal-MPI: Enabling Multi-Plane Images for Dynamic Scene Modelling  via Temporal Basis Learning

Many popular tourist landmarks are captured in a multitude of online, public
photos. These photos represent a sparse and unstructured sampling of the
plenoptic function for a particular scene. In this paper,we present a new
approach to novel view synthesis under time-varying illumination from such
data. Our approach builds on the recent multi-plane image (MPI) format for
representing local light fields under fixed viewing conditions. We introduce a
new DeepMPI representation, motivated by observations on the sparsity structure
of the plenoptic function, that allows for real-time synthesis of
photorealistic views that are continuous in both space and across changes in
lighting. Our method can synthesize the same compelling parallax and
view-dependent effects as previous MPI methods, while simultaneously
interpolating along changes in reflectance and illumination with time. We show
how to learn a model of these effects in an unsupervised way from an
unstructured collection of photos without temporal registration, demonstrating
significant improvements over recent work in neural rendering. More information
can be found crowdsampling.io.

本文介绍了一种基于多平面图像格式的新型视角合成方法，在不需要时间注册的情况下，从非结构化的照片集合中无监督地学习了这些效果的模型，并具有实时合成连续的空间和时间照明变化下逼真图像的能力，其中 DeepMPI 表现出了非常好的稀疏性能。