Recent research that combines implicit 3D representation with semantic
information, like Semantic-NeRF, has proven that NeRF model could perform
excellently in rendering 3D structures with semantic labels. This research aims
to extend the Semantic Neural Radiance Fields (Semantic-NeRF) model by focusing
solely on semantic output and removing the RGB output component. We reformulate
the model and its training procedure to leverage only the cross-entropy loss
between the model semantic output and the ground truth semantic images,
removing the colour data traditionally used in the original Semantic-NeRF
approach. We then conduct a series of identical experiments using the original
and the modified Semantic-NeRF model. Our primary objective is to obverse the
impact of this modification on the model performance by Semantic-NeRF, focusing
on tasks such as scene understanding, object detection, and segmentation. The
results offer valuable insights into the new way of rendering the scenes and
provide an avenue for further research and development in semantic-focused 3D
scene understanding.

优化 Semantic-NeRF 模型以实现仅有语义输出，并移除 RGB 输出成分，通过对比实验证明这种改动对于场景理解、目标检测和分割等任务的影响，提供了渲染场景的新方式并促进了进一步的研究和发展。

语义就够了：仅基于语义信息的 NeRF 重建

Semantic Is Enough: Only Semantic Information For NeRF Reconstruction

An increasing amount of applications rely on data-driven models that are
deployed for perception tasks across a sequence of scenes. Due to the mismatch
between training and deployment data, adapting the model on the new scenes is
often crucial to obtain good performance. In this work, we study continual
multi-scene adaptation for the task of semantic segmentation, assuming that no
ground-truth labels are available during deployment and that performance on the
previous scenes should be maintained. We propose training a Semantic-NeRF
network for each scene by fusing the predictions of a segmentation model and
then using the view-consistent rendered semantic labels as pseudo-labels to
adapt the model. Through joint training with the segmentation model, the
Semantic-NeRF model effectively enables 2D-3D knowledge transfer. Furthermore,
due to its compact size, it can be stored in a long-term memory and
subsequently used to render data from arbitrary viewpoints to reduce
forgetting. We evaluate our approach on ScanNet, where we outperform both a
voxel-based baseline and a state-of-the-art unsupervised domain adaptation
method.

本文研究了语义分割的连续多场景自适应问题，提出了基于 Semantic-NeRF 网络的解决方案，通过将分割模型的预测结果融合，使用视点一致的渲染语义标签来适应模型。我们在 ScanNet 数据集上测试，取得了比基线和最先进的无监督域自适应方法更好的性能。