We present NeSF, a method for producing 3D semantic fields from posed RGB
images alone. In place of classical 3D representations, our method builds on
recent work in implicit neural scene representations wherein 3D structure is
captured by point-wise functions. We leverage this methodology to recover 3D
density fields upon which we then train a 3D semantic segmentation model
supervised by posed 2D semantic maps. Despite being trained on 2D signals
alone, our method is able to generate 3D-consistent semantic maps from novel
camera poses and can be queried at arbitrary 3D points. Notably, NeSF is
compatible with any method producing a density field, and its accuracy improves
as the quality of the density field improves. Our empirical analysis
demonstrates comparable quality to competitive 2D and 3D semantic segmentation
baselines on complex, realistically rendered synthetic scenes. Our method is
the first to offer truly dense 3D scene segmentations requiring only 2D
supervision for training, and does not require any semantic input for inference
on novel scenes. We encourage the readers to visit the project website.

NeSF 是一种从 RGB 图像中生成 3D 语义场的方法，它利用了隐式神经场表示法，根据点积函数捕捉 3D 结构，能够在任意 3D 点上查询，并且生成可以与真实场景相对应的 3D 一致的语义图谱，是第一种只需要 2D 监督进行训练的真正密集的 3D 场景分割方法。