We study a new problem of semantic complete scene forecasting (SCSF) in this
work. Given a 4D dynamic point cloud sequence, our goal is to forecast the
complete scene corresponding to the future next frame along with its semantic
labels. To tackle this challenging problem, we properly model the synergetic
relationship between future forecasting and semantic scene completion through a
novel network named SCSFNet. SCSFNet leverages a hybrid geometric
representation for high-resolution complete scene forecasting. To leverage
multi-frame observation as well as the understanding of scene dynamics to ease
the completion task, SCSFNet introduces an attention-based skip connection
scheme. To ease the need to model occlusion variations and to better focus on
the occluded part, SCSFNet utilizes auxiliary visibility grids to guide the
forecasting task. To evaluate the effectiveness of SCSFNet, we conduct
experiments on various benchmarks including two large-scale indoor benchmarks
we contributed and the outdoor SemanticKITTI benchmark. Extensive experiments
show SCSFNet outperforms baseline methods on multiple metrics by a large
margin, and also prove the synergy between future forecasting and semantic
scene completion.

我们研究了语义完整场景预测（SCSF）的新问题，通过一个名为 SCSFNet 的新型网络，我们能够准确地预测未来帧的完整场景以及其语义标签。