Existing methods of cross-modal domain adaptation for 3D semantic
segmentation predict results only via 2D-3D complementarity that is obtained by
cross-modal feature matching. However, as lacking supervision in the target
domain, the complementarity is not always reliable. The results are not ideal
when the domain gap is large. To solve the problem of lacking supervision, we
introduce masked modeling into this task and propose a method Mx2M, which
utilizes masked cross-modality modeling to reduce the large domain gap. Our
Mx2M contains two components. One is the core solution, cross-modal removal and
prediction (xMRP), which makes the Mx2M adapt to various scenarios and provides
cross-modal self-supervision. The other is a new way of cross-modal feature
matching, the dynamic cross-modal filter (DxMF) that ensures the whole method
dynamically uses more suitable 2D-3D complementarity. Evaluation of the Mx2M on
three DA scenarios, including Day/Night, USA/Singapore, and A2D2/SemanticKITTI,
brings large improvements over previous methods on many metrics.

现有的跨模态域自适应方法在三维语义分割中仅通过跨模态特征匹配预测结果的 2D-3D 补充性，然而，在目标领域缺乏监督的情况下，这种补充性并不总是可靠的。我们为解决缺乏监督的问题，将蒙版建模引入该任务，并提出了一种名为 Mx2M 的方法，该方法利用蒙版的跨模态建模来减小域间差距。我们的 Mx2M 包含两个组成部分。一个是核心解决方案，即跨模态去除和预测（xMRP），它使得 Mx2M 适应各种情景并提供跨模态自监督。另一个是一种新的跨模态特征匹配方式，即动态跨模态滤波器（DxMF），它确保整个方法动态地使用更合适的 2D-3D 补充性。在包括白天 / 夜晚、美国 / 新加坡和 A2D2/SemanticKITTI 在内的三个跨模态自适应场景上，对 Mx2M 的评估在许多指标上取得了显著的改进。