State-of-the-art multimodal semantic segmentation approaches combining LiDAR
and color data are usually designed on top of asymmetric information-sharing
schemes and assume that both modalities are always available. Regrettably, this
strong assumption may not hold in real-world scenarios, where sensors are prone
to failure or can face adverse conditions (night-time, rain, fog, etc.) that
make the acquired information unreliable. Moreover, these architectures tend to
fail in continual learning scenarios. In this work, we re-frame the task of
multimodal semantic segmentation by enforcing a tightly-coupled feature
representation and a symmetric information-sharing scheme, which allows our
approach to work even when one of the input modalities is missing. This makes
our model reliable even in safety-critical settings, as is the case of
autonomous driving. We evaluate our approach on the SemanticKITTI dataset,
comparing it with our closest competitor. We also introduce an ad-hoc continual
learning scheme and show results in a class-incremental continual learning
scenario that prove the effectiveness of the approach also in this setting.

本研究通过强制执行紧密耦合的特征表示和对称信息共享方案，通过重新定义多模态语义分割任务，实现即使一个输入模态缺失，也能正常工作，提高模型的可靠性，尤其适用于自动驾驶等安全关键应用领域。在 SemanticKITTI 数据集上评估了本方法，并与最接近的竞争者进行了比较。同时还引入了一种特定的持续学习方案，并在渐进式持续学习场景中展示了该方法的有效性。