We propose a late-to-early recurrent feature fusion scheme for 3D object
detection using temporal LiDAR point clouds. Our main motivation is fusing
object-aware latent embeddings into the early stages of a 3D object detector.
This feature fusion strategy enables the model to better capture the shapes and
poses for challenging objects, compared with learning from raw points directly.
Our method conducts late-to-early feature fusion in a recurrent manner. This is
achieved by enforcing window-based attention blocks upon temporally calibrated
and aligned sparse pillar tokens. Leveraging bird's eye view foreground pillar
segmentation, we reduce the number of sparse history features that our model
needs to fuse into its current frame by 10$\times$. We also propose a
stochastic-length FrameDrop training technique, which generalizes the model to
variable frame lengths at inference for improved performance without
retraining. We evaluate our method on the widely adopted Waymo Open Dataset and
demonstrate improvement on 3D object detection against the baseline model,
especially for the challenging category of large objects.

我们提出了一种基于时间 LiDAR 点云的迟到提早循环特征融合方案，用于 3D 物体检测。我们的主要动机是将物体感知的潜在嵌入融合到 3D 物体检测器的早期阶段。这种特征融合策略使模型能够更好地捕捉具有挑战性的物体的形状和姿态，相比直接从原始点学习。我们的方法以循环的方式进行迟到提早特征融合。这是通过在时间上校准和对齐的稀疏柱状令牌上施加基于窗口的注意力块来实现的。利用俯视图前景柱状分割，我们将模型需要融合到当前帧中的稀疏历史特征数量减少了 10 倍。我们还提出了一种随机长度的 FrameDrop 训练技术，它在推断时可以依据可变帧长度来改进性能而无需重新训练。我们在广泛采用的 Waymo Open Dataset 上评估了我们的方法，并证明在 3D 物体检测方面相对于基线模型有所改进，特别是对于挑战性的大物体类别。