Over the past years, we have observed an abundance of approaches for modeling
dynamic 3D scenes using Gaussian Splatting (GS). Such solutions use GS to
represent the scene's structure and the neural network to model dynamics. Such
approaches allow fast rendering and extracting each element of such a dynamic
scene. However, modifying such objects over time is challenging. SC-GS (Sparse
Controlled Gaussian Splatting) enhanced with Deformed Control Points partially
solves this issue. However, this approach necessitates selecting elements that
need to be kept fixed, as well as centroids that should be adjusted throughout
editing. Moreover, this task poses additional difficulties regarding the
re-productivity of such editing. To address this, we propose Dynamic
Multi-Gaussian Soup (D-MiSo), which allows us to model the mesh-inspired
representation of dynamic GS. Additionally, we propose a strategy of linking
parameterized Gaussian splats, forming a Triangle Soup with the estimated mesh.
Consequently, we can separately construct new trajectories for the 3D objects
composing the scene. Thus, we can make the scene's dynamic editable over time
or while maintaining partial dynamics.

通过提出动态多高斯汤（D-MiSo）方法和参数化高斯斑点的连接策略来对动态 3D 场景进行建模，实现对场景的局部动态编辑，解决了高斯飞溅模型存在的时间动态修改的难题。

D-MiSo：使用多高斯混合体编辑动态 3D 场景

D-MiSo: Editing Dynamic 3D Scenes using Multi-Gaussians Soup

In this paper, we propose the Masked Space-Time Hash encoding (MSTH), a novel
method for efficiently reconstructing dynamic 3D scenes from multi-view or
monocular videos. Based on the observation that dynamic scenes often contain
substantial static areas that result in redundancy in storage and computations,
MSTH represents a dynamic scene as a weighted combination of a 3D hash encoding
and a 4D hash encoding. The weights for the two components are represented by a
learnable mask which is guided by an uncertainty-based objective to reflect the
spatial and temporal importance of each 3D position. With this design, our
method can reduce the hash collision rate by avoiding redundant queries and
modifications on static areas, making it feasible to represent a large number
of space-time voxels by hash tables with small size.Besides, without the
requirements to fit the large numbers of temporally redundant features
independently, our method is easier to optimize and converge rapidly with only
twenty minutes of training for a 300-frame dynamic scene.As a result, MSTH
obtains consistently better results than previous methods with only 20 minutes
of training time and 130 MB of memory storage. Code is available at
this https URL

提出了一种名为 “Masked Space-Time Hash encoding (MSTH)” 的新方法，用于高效地从多视角或单目视频中重建动态 3D 场景。该方法通过将动态场景表示为 3D 哈希编码和 4D 哈希编码的加权组合，通过可学习的遮罩来表示两个组件的权重，以反映每个 3D 位置的空间和时间重要性。通过避免对静态区域进行冗余查询和修改，该方法可以降低哈希碰撞率，使得能够使用小尺寸的哈希表来表示大量的时空体素。此外，该方法不需要独立地适应大量的时态冗余特征，易于优化，并且仅需 20 分钟的训练时间即可收敛。MSTH 方法在只有 20 分钟的训练时间和 130 MB 的内存存储的情况下，始终比以前的方法表现更好。

用于高效动态场景重建的掩码空时哈希编码

Masked Space-Time Hash Encoding for Efficient Dynamic Scene  Reconstruction

Neural fields, a category of neural networks trained to represent
high-frequency signals, have gained significant attention in recent years due
to their impressive performance in modeling complex 3D data, especially large
neural signed distance (SDFs) or radiance fields (NeRFs) via a single
multi-layer perceptron (MLP). However, despite the power and simplicity of
representing signals with an MLP, these methods still face challenges when
modeling large and complex temporal signals due to the limited capacity of
MLPs. In this paper, we propose an effective approach to address this
limitation by incorporating temporal residual layers into neural fields, dubbed
ResFields, a novel class of networks specifically designed to effectively
represent complex temporal signals. We conduct a comprehensive analysis of the
properties of ResFields and propose a matrix factorization technique to reduce
the number of trainable parameters and enhance generalization capabilities.
Importantly, our formulation seamlessly integrates with existing techniques and
consistently improves results across various challenging tasks: 2D video
approximation, dynamic shape modeling via temporal SDFs, and dynamic NeRF
reconstruction. Lastly, we demonstrate the practical utility of ResFields by
showcasing its effectiveness in capturing dynamic 3D scenes from sparse sensory
inputs of a lightweight capture system.

提出了一种将时间残差层引入神经场的有效方法，用于表示复杂的时间信号，通过降低可训练参数数量和增强泛化能力的矩阵分解技术，实现了对 2D 视频逼近、通过时间 SDFs 进行动态形状建模和动态 NeRF 重建等各种具有挑战性任务中结果的稳定提升，并通过展示其在轻量级捕获系统的稀疏传感输入中捕获动态 3D 场景的实际效果证明了 ResFields 的实用性。

ResFields：用于时空信号的残差神经场

ResFields: Residual Neural Fields for Spatiotemporal Signals

Modeling and re-rendering dynamic 3D scenes is a challenging task in 3D
vision. Prior approaches build on NeRF and rely on implicit representations.
This is slow since it requires many MLP evaluations, constraining real-world
applications. We show that dynamic 3D scenes can be explicitly represented by
six planes of learned features, leading to an elegant solution we call
HexPlane. A HexPlane computes features for points in spacetime by fusing
vectors extracted from each plane, which is highly efficient. Pairing a
HexPlane with a tiny MLP to regress output colors and training via volume
rendering gives impressive results for novel view synthesis on dynamic scenes,
matching the image quality of prior work but reducing training time by more
than $100\times$. Extensive ablations confirm our HexPlane design and show that
it is robust to different feature fusion mechanisms, coordinate systems, and
decoding mechanisms. HexPlane is a simple and effective solution for
representing 4D volumes, and we hope they can broadly contribute to modeling
spacetime for dynamic 3D scenes.

通过学习特征的六个平面对动态 3D 场景的显式建模，开发了一种高效的解决方案，称为 HexPlane，在节约训练时间的同时，能够在视角合成方面取得与之前研究相匹配的结果。