Understanding complex scenes at multiple levels of abstraction remains a
formidable challenge in computer vision. To address this, we introduce Nested
Neural Feature Fields (N2F2), a novel approach that employs hierarchical
supervision to learn a single feature field, wherein different dimensions
within the same high-dimensional feature encode scene properties at varying
granularities. Our method allows for a flexible definition of hierarchies,
tailored to either the physical dimensions or semantics or both, thereby
enabling a comprehensive and nuanced understanding of scenes. We leverage a 2D
class-agnostic segmentation model to provide semantically meaningful pixel
groupings at arbitrary scales in the image space, and query the CLIP
vision-encoder to obtain language-aligned embeddings for each of these
segments. Our proposed hierarchical supervision method then assigns different
nested dimensions of the feature field to distill the CLIP embeddings using
deferred volumetric rendering at varying physical scales, creating a
coarse-to-fine representation. Extensive experiments show that our approach
outperforms the state-of-the-art feature field distillation methods on tasks
such as open-vocabulary 3D segmentation and localization, demonstrating the
effectiveness of the learned nested feature field.

通过引入分层监督和延迟体积渲染，我们提出了一种名为 Nested Neural Feature Fields（N2F2）的新方法，通过学习单个特征场来对复杂场景进行多层次的抽象理解，实现了对场景的全面和细致的理解，取得了在计算机视觉领域的一些任务上超越了现有方法。