We propose a late-to-early recurrent feature fusion scheme for 3D object detection using temporal LiDAR point clouds. Our main motivation is fusing object-aware latent embeddings into the early stages of a 3D object detector. This feature fusion strategy enables the model to better capture the shapes and poses for challenging objects, compared with learning from raw points directly. Our method conducts late-to-early feature fusion in a recurrent manner. This is achieved by enforcing window-based attention blocks upon temporally calibrated and aligned sparse pillar tokens. Leveraging bird's eye view foreground pillar segmentation, we reduce the number of sparse history features that our model needs to fuse into its current frame by 10$\times$. We also propose a stochastic-length FrameDrop training technique, which generalizes the model to variable frame lengths at inference for improved performance without retraining. We evaluate our method on the widely adopted Waymo Open Dataset and demonstrate improvement on 3D object detection against the baseline model, especially for the challenging category of large objects.

我们提出了一种基于时间LiDAR点云的迟到提早循环特征融合方案，用于3D物体检测。我们的主要动机是将物体感知的潜在嵌入融合到3D物体检测器的早期阶段。这种特征融合策略使模型能够更好地捕捉具有挑战性的物体的形状和姿态，相比直接从原始点学习。我们的方法以循环的方式进行迟到提早特征融合。这是通过在时间上校准和对齐的稀疏柱状令牌上施加基于窗口的注意力块来实现的。利用俯视图前景柱状分割，我们将模型需要融合到当前帧中的稀疏历史特征数量减少了10倍。我们还提出了一种随机长度的FrameDrop训练技术，它在推断时可以依据可变帧长度来改进性能而无需重新训练。我们在广泛采用的Waymo Open Dataset上评估了我们的方法，并证明在3D物体检测方面相对于基线模型有所改进，特别是对于挑战性的大物体类别。

LEF: 后期-早期时序融合在LiDAR 3D目标检测中的应用