4D panoptic segmentation is a challenging but practically useful task that
requires every point in a LiDAR point-cloud sequence to be assigned a semantic
class label, and individual objects to be segmented and tracked over time.
Existing approaches utilize only LiDAR inputs which convey limited information
in regions with point sparsity. This problem can, however, be mitigated by
utilizing RGB camera images which offer appearance-based information that can
reinforce the geometry-based LiDAR features. Motivated by this, we propose
4D-Former: a novel method for 4D panoptic segmentation which leverages both
LiDAR and image modalities, and predicts semantic masks as well as temporally
consistent object masks for the input point-cloud sequence. We encode semantic
classes and objects using a set of concise queries which absorb feature
information from both data modalities. Additionally, we propose a learned
mechanism to associate object tracks over time which reasons over both
appearance and spatial location. We apply 4D-Former to the nuScenes and
SemanticKITTI datasets where it achieves state-of-the-art results.

通过整合 LiDAR 和图像数据，4D-Former 方法在 4D 全景分割任务上取得了最先进的结果，能够为 LiDAR 序列中的每个点分配语义类别标签和进行对象分割，并能在时间上保持一致的对象分割遮罩。