Synthesizing photo-realistic images and videos is at the heart of computer
graphics and has been the focus of decades of research. Traditionally,
synthetic images of a scene are generated using rendering algorithms such as
rasterization or ray tracing, which take specifically defined representations
of geometry and material properties as input. Collectively, these inputs define
the actual scene and what is rendered, and are referred to as the scene
representation (where a scene consists of one or more objects). Example scene
representations are triangle meshes with accompanied textures (e.g., created by
an artist), point clouds (e.g., from a depth sensor), volumetric grids (e.g.,
from a CT scan), or implicit surface functions (e.g., truncated signed distance
fields). The reconstruction of such a scene representation from observations
using differentiable rendering losses is known as inverse graphics or inverse
rendering. Neural rendering is closely related, and combines ideas from
classical computer graphics and machine learning to create algorithms for
synthesizing images from real-world observations. Neural rendering is a leap
forward towards the goal of synthesizing photo-realistic image and video
content. In recent years, we have seen immense progress in this field through
hundreds of publications that show different ways to inject learnable
components into the rendering pipeline. This state-of-the-art report on
advances in neural rendering focuses on methods that combine classical
rendering principles with learned 3D scene representations, often now referred
to as neural scene representations. A key advantage of these methods is that
they are 3D-consistent by design, enabling applications such as novel viewpoint
synthesis of a captured scene. In addition to methods that handle static
scenes, we cover neural scene representations for modeling non-rigidly
deforming objects...

该论文综述了近年来神经渲染领域的进展，重点介绍了将经典渲染原则与学习得到的 3D 场景表示相结合的方法，包括静态场景处理和非刚性变形物体建模等。