It has long been challenging to recover the underlying dynamic 3D scene representations from a monocular RGB video. Existing works formulate this problem into finding a single most plausible solution by adding various constraints such as depth priors and strong geometry constraints, ignoring the fact that there could be infinitely many 3D scene representations corresponding to a single dynamic video. In this paper, we aim to learn all plausible 3D scene configurations that match the input video, instead of just inferring a specific one. To achieve this ambitious goal, we introduce a new framework, called OSN. The key to our approach is a simple yet innovative object scale network together with a joint optimization module to learn an accurate scale range for every dynamic 3D object. This allows us to sample as many faithful 3D scene configurations as possible. Extensive experiments show that our method surpasses all baselines and achieves superior accuracy in dynamic novel view synthesis on multiple synthetic and real-world datasets. Most notably, our method demonstrates a clear advantage in learning fine-grained 3D scene geometry. Our code and data are available at https://github.com/vLAR-group/OSN

通过引入OSN框架，本研究旨在学习与输入视频匹配的所有可能的三维场景配置，而不仅仅推断出一种具体的解决方案，通过使用简单而创新的对象尺度网络和联合优化模块来学习每个动态三维物体的准确尺度范围，从而尽可能采样多个忠实的三维场景配置，并通过在多个合成和真实数据集上进行广泛实验证明了我们的方法在动态新视图合成中超越了所有基线，并实现了更高的准确性，尤其在学习细粒度的三维场景几何中展现出明显优势。

OSN：来自单目视频的动态3D场景的无限表示