Understanding 3D scenes is a critical prerequisite for autonomous agents.
Recently, LiDAR and other sensors have made large amounts of data available in
the form of temporal sequences of point cloud frames. In this work, we propose
a novel problem -- sequential scene flow estimation (SSFE) -- that aims to
predict 3D scene flow for all pairs of point clouds in a given sequence. This
is unlike the previously studied problem of scene flow estimation which focuses
on two frames.
We introduce the SPCM-Net architecture, which solves this problem by
computing multi-scale spatiotemporal correlations between neighboring point
clouds and then aggregating the correlation across time with an order-invariant
recurrent unit. Our experimental evaluation confirms that recurrent processing
of point cloud sequences results in significantly better SSFE compared to using
only two frames. Additionally, we demonstrate that this approach can be
effectively modified for sequential point cloud forecasting (SPF), a related
problem that demands forecasting future point cloud frames.
Our experimental results are evaluated using a new benchmark for both SSFE
and SPF consisting of synthetic and real datasets. Previously, datasets for
scene flow estimation have been limited to two frames. We provide non-trivial
extensions to these datasets for multi-frame estimation and prediction. Due to
the difficulty of obtaining ground truth motion for real-world datasets, we use
self-supervised training and evaluation metrics. We believe that this benchmark
will be pivotal to future research in this area. All code for benchmark and
models will be made accessible.

提出了一个新问题 -- 顺序场景流估计（SSFE），旨在从给定序列中估计所有点云对的三维场景流。引入 SPCM-Net 架构来解决该问题，利用时间和空间的多尺度相互关系以及顺序不变的循环单元来聚合关联。通过实验验证表明，与仅使用两个帧相比，处理点云序列的递归处理结果更好。这一方法可有效地修改为顺序点云预测（SPF）的问题，并使用自监督训练和评估指标进行实验结果的评估。提供了新的基准数据集，对多帧估计和预测进行测试，这将在未来的研究中起到重要作用。