Existing LiDAR-based 3D object detectors usually focus on the single-frame
detection, while ignoring the spatiotemporal information in consecutive point
cloud frames. In this paper, we propose an end-to-end online 3D video object
detector that operates on point cloud sequences. The proposed model comprises a
spatial feature encoding component and a spatiotemporal feature aggregation
component. In the former component, a novel Pillar Message Passing Network
(PMPNet) is proposed to encode each discrete point cloud frame. It adaptively
collects information for a pillar node from its neighbors by iterative message
passing, which effectively enlarges the receptive field of the pillar feature.
In the latter component, we propose an Attentive Spatiotemporal Transformer GRU
(AST-GRU) to aggregate the spatiotemporal information, which enhances the
conventional ConvGRU with an attentive memory gating mechanism. AST-GRU
contains a Spatial Transformer Attention (STA) module and a Temporal
Transformer Attention (TTA) module, which can emphasize the foreground objects
and align the dynamic objects, respectively. Experimental results demonstrate
that the proposed 3D video object detector achieves state-of-the-art
performance on the large-scale nuScenes benchmark.

本文提出了一个端到端的在线三维视频物体检测器，该模型由空间特征编码组件和时空特征聚合组件组成。在空间特征编码组件中，提出了一种新的 Pillar Message Passing Network（PMPNet）模型，用于对离散点云帧进行编码。在时空特征聚合组件中，提出了一种注意力时空变换门控循环神经网络（AST-GRU），它通过注意力机制强调了前景物体并对动态物体进行了对齐。实验结果表明，该模型在大规模 nuScenes 基准测试中实现了最先进的性能。