Dynamic reconstruction and spatiotemporal novel-view synthesis of non-rigidly
deforming scenes recently gained increased attention. While existing work
achieves impressive quality and performance on multi-view or teleporting camera
setups, most methods fail to efficiently and faithfully recover motion and
appearance from casual monocular captures. This paper contributes to the field
by introducing a new method for dynamic novel view synthesis from monocular
video, such as casual smartphone captures.
Our approach represents the scene as a $\textit{dynamic neural point cloud}$,
an implicit time-conditioned point distribution that encodes local geometry and
appearance in separate hash-encoded neural feature grids for static and dynamic
regions. By sampling a discrete point cloud from our model, we can efficiently
render high-quality novel views using a fast differentiable rasterizer and
neural rendering network. Similar to recent work, we leverage advances in
neural scene analysis by incorporating data-driven priors like monocular depth
estimation and object segmentation to resolve motion and depth ambiguities
originating from the monocular captures. In addition to guiding the
optimization process, we show that these priors can be exploited to explicitly
initialize our scene representation to drastically improve optimization speed
and final image quality. As evidenced by our experimental evaluation, our
dynamic point cloud model not only enables fast optimization and real-time
frame rates for interactive applications, but also achieves competitive image
quality on monocular benchmark sequences.
Our project page is available at
this https URL

该研究论文介绍了一种从单目视频中进行动态新视图合成的新方法，通过将场景表示为动态神经点云，利用数据驱动的先验信息和快速可微分光栅化器和神经渲染网络，实现高质量的合成图像。

D-NPC: 动态神经点云用于从单目视频进行非刚性视图合成

D-NPC: Dynamic Neural Point Clouds for Non-Rigid View Synthesis from  Monocular Video

Although neural radiance fields (NeRFs) have achieved triumphs in image novel
view synthesis (NVS), LiDAR NVS remains largely unexplored. Previous LiDAR NVS
methods employ a simple shift from image NVS methods while ignoring the dynamic
nature and the large-scale reconstruction problem of LiDAR point clouds. In
light of this, we propose LiDAR4D, a differentiable LiDAR-only framework for
novel space-time LiDAR view synthesis. In consideration of the sparsity and
large-scale characteristics, we design a 4D hybrid representation combined with
multi-planar and grid features to achieve effective reconstruction in a
coarse-to-fine manner. Furthermore, we introduce geometric constraints derived
from point clouds to improve temporal consistency. For the realistic synthesis
of LiDAR point clouds, we incorporate the global optimization of ray-drop
probability to preserve cross-region patterns. Extensive experiments on
KITTI-360 and NuScenes datasets demonstrate the superiority of our method in
accomplishing geometry-aware and time-consistent dynamic reconstruction. Codes
are available at this https URL

基于 LiDAR 的 LiDAR4D 框架，通过 4D 混合表示和几何限制，实现了具有几何感和时间一致性的动态重构技术。

LiDAR4D: 动态神经场用于新颖时空视角 LiDAR 合成

LiDAR4D: Dynamic Neural Fields for Novel Space-time View LiDAR Synthesis

Dynamic reconstruction with neural radiance fields (NeRF) requires accurate
camera poses. These are often hard to retrieve with existing
structure-from-motion (SfM) pipelines as both camera and scene content can
change. We propose DynaMoN that leverages simultaneous localization and mapping
(SLAM) jointly with motion masking to handle dynamic scene content. Our robust
SLAM-based tracking module significantly accelerates the training process of
the dynamic NeRF while improving the quality of synthesized views at the same
time. Extensive experimental validation on TUM RGB-D, BONN RGB-D Dynamic and
the DyCheck's iPhone dataset, three real-world datasets, shows the advantages
of DynaMoN both for camera pose estimation and novel view synthesis.

使用动态重建和神经辐射场 (NeRF) 时，准确的摄像机姿态往往难以使用现有的视觉结构重建 (SfM) 流程获取，因为摄像机和场景内容都可能发生变化。我们提出了 DynaMoN，它与运动遮罩相结合来处理动态场景内容的同时定位与建图 (SLAM)。我们基于稳健的基于 SLAM 的跟踪模块显著加速了动态 NeRF 的训练过程，并同时提高了合成视图的质量。大规模的实验验证（在 TUM RGB-D、BONN RGB-D Dynamic 和 DyCheck 的 iPhone 数据集上）显示了 DynaMoN 在摄像机姿态估计和新视图合成方面的优势。

DynaMoN：动态 NeRF 的快速可靠的相机定位

DynaMoN: Motion-Aware Fast And Robust Camera Localization for Dynamic  NeRF

We present Tensor4D, an efficient yet effective approach to dynamic scene
modeling. The key of our solution is an efficient 4D tensor decomposition
method so that the dynamic scene can be directly represented as a 4D
spatio-temporal tensor. To tackle the accompanying memory issue, we decompose
the 4D tensor hierarchically by projecting it first into three time-aware
volumes and then nine compact feature planes. In this way, spatial information
over time can be simultaneously captured in a compact and memory-efficient
manner. When applying Tensor4D for dynamic scene reconstruction and rendering,
we further factorize the 4D fields to different scales in the sense that
structural motions and dynamic detailed changes can be learned from coarse to
fine. The effectiveness of our method is validated on both synthetic and
real-world scenes. Extensive experiments show that our method is able to
achieve high-quality dynamic reconstruction and rendering from sparse-view
camera rigs or even a monocular camera. The code and dataset will be released
at this https URL

Tensor4D 是一种高效而有效的动态场景建模方法，通过 4D 张量分解来直接表示动态场景，并通过不同尺度上的分解进一步实现动态构建和渲染。在合成和现实世界场景中验证了其效果，具有高质量的动态重建和渲染功能。