In this paper, we propose the Masked Space-Time Hash encoding (MSTH), a novel
method for efficiently reconstructing dynamic 3D scenes from multi-view or
monocular videos. Based on the observation that dynamic scenes often contain
substantial static areas that result in redundancy in storage and computations,
MSTH represents a dynamic scene as a weighted combination of a 3D hash encoding
and a 4D hash encoding. The weights for the two components are represented by a
learnable mask which is guided by an uncertainty-based objective to reflect the
spatial and temporal importance of each 3D position. With this design, our
method can reduce the hash collision rate by avoiding redundant queries and
modifications on static areas, making it feasible to represent a large number
of space-time voxels by hash tables with small size.Besides, without the
requirements to fit the large numbers of temporally redundant features
independently, our method is easier to optimize and converge rapidly with only
twenty minutes of training for a 300-frame dynamic scene.As a result, MSTH
obtains consistently better results than previous methods with only 20 minutes
of training time and 130 MB of memory storage. Code is available at
this https URL

提出了一种名为 “Masked Space-Time Hash encoding (MSTH)” 的新方法，用于高效地从多视角或单目视频中重建动态 3D 场景。该方法通过将动态场景表示为 3D 哈希编码和 4D 哈希编码的加权组合，通过可学习的遮罩来表示两个组件的权重，以反映每个 3D 位置的空间和时间重要性。通过避免对静态区域进行冗余查询和修改，该方法可以降低哈希碰撞率，使得能够使用小尺寸的哈希表来表示大量的时空体素。此外，该方法不需要独立地适应大量的时态冗余特征，易于优化，并且仅需 20 分钟的训练时间即可收敛。MSTH 方法在只有 20 分钟的训练时间和 130 MB 的内存存储的情况下，始终比以前的方法表现更好。