The Detection Transformer (DETR) has revolutionized the design of CNN-based
object detection systems, showcasing impressive performance. However, its
potential in the domain of multi-frame 3D object detection remains largely
unexplored. In this paper, we present STEMD, a novel end-to-end framework for
multi-frame 3D object detection based on the DETR-like paradigm. Our approach
treats multi-frame 3D object detection as a sequence-to-sequence task and
effectively captures spatial-temporal dependencies at both the feature and
query levels. To model the inter-object spatial interaction and complex
temporal dependencies, we introduce the spatial-temporal graph attention
network. This network represents queries as nodes in a graph and enables
effective modeling of object interactions within a social context. In addition,
to solve the problem of missing hard cases in the proposed output of the
encoder in the current frame, we incorporate the output of the previous frame
to initialize the query input of the decoder. Moreover, we tackle the issue of
redundant detection results, where the model generates numerous overlapping
boxes from similar queries. To mitigate this, we introduce an IoU
regularization term in the loss function. This term aids in distinguishing
between queries matched with the ground-truth box and queries that are similar
but unmatched during the refinement process, leading to reduced redundancy and
more accurate detections. Through extensive experiments, we demonstrate the
effectiveness of our approach in handling challenging scenarios, while
incurring only a minor additional computational overhead. The code will be
available at https://github.com/Eaphan/STEMD.

STEMD 是一种基于 DETR 的新型端到端框架，用于实现多帧 3D 物体检测，其有效地捕获了特征和相应级别的时空依赖性，并引入了空间 - 时间图注意力网络，以解决对象之间的空间交互作用和复杂的时间依赖性问题，控制冗余检测结果。