An increasing amount of applications rely on data-driven models that are
deployed for perception tasks across a sequence of scenes. Due to the mismatch
between training and deployment data, adapting the model on the new scenes is
often crucial to obtain good performance. In this work, we study continual
multi-scene adaptation for the task of semantic segmentation, assuming that no
ground-truth labels are available during deployment and that performance on the
previous scenes should be maintained. We propose training a Semantic-NeRF
network for each scene by fusing the predictions of a segmentation model and
then using the view-consistent rendered semantic labels as pseudo-labels to
adapt the model. Through joint training with the segmentation model, the
Semantic-NeRF model effectively enables 2D-3D knowledge transfer. Furthermore,
due to its compact size, it can be stored in a long-term memory and
subsequently used to render data from arbitrary viewpoints to reduce
forgetting. We evaluate our approach on ScanNet, where we outperform both a
voxel-based baseline and a state-of-the-art unsupervised domain adaptation
method.

本文研究了语义分割的连续多场景自适应问题，提出了基于 Semantic-NeRF 网络的解决方案，通过将分割模型的预测结果融合，使用视点一致的渲染语义标签来适应模型。我们在 ScanNet 数据集上测试，取得了比基线和最先进的无监督域自适应方法更好的性能。