The semantically interactive radiance field has always been an appealing task
for its potential to facilitate user-friendly and automated real-world 3D scene
understanding applications. However, it is a challenging task to achieve high
quality, efficiency and zero-shot ability at the same time with semantics in
radiance fields. In this work, we present FastLGS, an approach that supports
real-time open-vocabulary query within 3D Gaussian Splatting (3DGS) under high
resolution. We propose the semantic feature grid to save multi-view CLIP
features which are extracted based on Segment Anything Model (SAM) masks, and
map the grids to low dimensional features for semantic field training through
3DGS. Once trained, we can restore pixel-aligned CLIP embeddings through
feature grids from rendered features for open-vocabulary queries. Comparisons
with other state-of-the-art methods prove that FastLGS can achieve the first
place performance concerning both speed and accuracy, where FastLGS is 98x
faster than LERF and 4x faster than LangSplat. Meanwhile, experiments show that
FastLGS is adaptive and compatible with many downstream tasks, such as 3D
segmentation and 3D object inpainting, which can be easily applied to other 3D
manipulation systems.

快速多语种查询 3D 场景的实时系统