We propose the problem of point-level 3D scene interpolation, which aims to
simultaneously reconstruct a 3D scene in two states from multiple views,
synthesize smooth point-level interpolations between them, and render the scene
from novel viewpoints, all without any supervision between the states. The
primary challenge is on achieving a smooth transition between states that may
involve significant and non-rigid changes. To address these challenges, we
introduce "PAPR in Motion", a novel approach that builds upon the recent
Proximity Attention Point Rendering (PAPR) technique, which can deform a point
cloud to match a significantly different shape and render a visually coherent
scene even after non-rigid deformations. Our approach is specifically designed
to maintain the temporal consistency of the geometric structure by introducing
various regularization techniques for PAPR. The result is a method that can
effectively bridge large scene changes and produce visually coherent and
temporally smooth interpolations in both geometry and appearance. Evaluation
across diverse motion types demonstrates that "PAPR in Motion" outperforms the
leading neural renderer for dynamic scenes. For more results and code, please
visit our project website at this https URL .

我们提出了点级三维场景插值的问题，旨在同时从多个视角重建两个状态的三维场景，合成它们之间的平滑点级插值，并以新视角渲染场景，而无需任何状态之间的监督。

PAPR 在运动中：无缝点级 3D 场景插值

PAPR in Motion: Seamless Point-level 3D Scene Interpolation

Advances in 3D reconstruction have enabled high-quality 3D capture, but
require a user to collect hundreds to thousands of images to create a 3D scene.
We present CAT3D, a method for creating anything in 3D by simulating this
real-world capture process with a multi-view diffusion model. Given any number
of input images and a set of target novel viewpoints, our model generates
highly consistent novel views of a scene. These generated views can be used as
input to robust 3D reconstruction techniques to produce 3D representations that
can be rendered from any viewpoint in real-time. CAT3D can create entire 3D
scenes in as little as one minute, and outperforms existing methods for single
image and few-view 3D scene creation. See our project page for results and
interactive demos at this https URL .

用于高质量 3D 捕捉的 CAT3D 方法模拟实际捕捉过程中的多视图扩散模型，通过给定任意数量的输入图像和一组目标新视点，生成高度一致的场景新视图，可用作强大的 3D 重建技术的输入，以实时渲染任意视点的 3D 表示。CAT3D 在短短一分钟内就可以创建完整的 3D 场景，并优于现有的单张图片和少视图 3D 场景创建方法。

CAT3D: 利用多视角扩散模型创建三维任何物体

CAT3D: Create Anything in 3D with Multi-View Diffusion Models

Rendering scenes observed in a monocular video from novel viewpoints is a
challenging problem. For static scenes the community has studied both
scene-specific optimization techniques, which optimize on every test scene, and
generalized techniques, which only run a deep net forward pass on a test scene.
In contrast, for dynamic scenes, scene-specific optimization techniques exist,
but, to our best knowledge, there is currently no generalized method for
dynamic novel view synthesis from a given monocular video. To answer whether
generalized dynamic novel view synthesis from monocular videos is possible
today, we establish an analysis framework based on existing techniques and work
toward the generalized approach. We find a pseudo-generalized process without
scene-specific appearance optimization is possible, but geometrically and
temporally consistent depth estimates are needed. Despite no scene-specific
appearance optimization, the pseudo-generalized approach improves upon some
scene-specific methods.

基于现有技术，本研究建立了一种分析框架，并致力于从给定的单目视频中实现广义动态新视角合成。通过伪广义方法，改进了某些场景特定方法，尽管没有场景特定外观优化。

单目视频中的广义动态新视角合成今天是否可能？

Is Generalized Dynamic Novel View Synthesis from Monocular Videos  Possible Today?

We present a method that takes as input a set of images of a scene
illuminated by unconstrained known lighting, and produces as output a 3D
representation that can be rendered from novel viewpoints under arbitrary
lighting conditions. Our method represents the scene as a continuous volumetric
function parameterized as MLPs whose inputs are a 3D location and whose outputs
are the following scene properties at that input location: volume density,
surface normal, material parameters, distance to the first surface intersection
in any direction, and visibility of the external environment in any direction.
Together, these allow us to render novel views of the object under arbitrary
lighting, including indirect illumination effects. The predicted visibility and
surface intersection fields are critical to our model's ability to simulate
direct and indirect illumination during training, because the brute-force
techniques used by prior work are intractable for lighting conditions outside
of controlled setups with a single light. Our method outperforms alternative
approaches for recovering relightable 3D scene representations, and performs
well in complex lighting settings that have posed a significant challenge to
prior work.

该研究提供了一种基于连续体积参数化的方法，可以将场景表示为可以在新视点下进行渲染的 3D 表示，并且可在任意照明条件下呈现，包括间接光照效果，从而解决了以前的方法因光照环境变化而受限的问题。