Vision Transformers have shown promising progress in various object detection
tasks, including monocular 2D/3D detection and surround-view 3D detection.
However, when used in essential and classic stereo 3D object detection,
directly adopting those surround-view Transformers leads to slow convergence
and significant precision drops. We argue that one of the causes of this defect
is that the surround-view Transformers do not consider the stereo-specific
image correspondence information. In a surround-view system, the overlapping
areas are small, and thus correspondence is not a primary issue. In this paper,
we explore the model design of vision Transformers in stereo 3D object
detection, focusing particularly on extracting and encoding the task-specific
image correspondence information. To achieve this goal, we present TS3D, a
Transformer-based Stereo-aware 3D object detector. In the TS3D, a
Disparity-Aware Positional Encoding (DAPE) model is proposed to embed the image
correspondence information into stereo features. The correspondence is encoded
as normalized disparity and is used in conjunction with sinusoidal 2D
positional encoding to provide the location information of the 3D scene. To
extract enriched multi-scale stereo features, we propose a Stereo Reserving
Feature Pyramid Network (SRFPN). The SRFPN is designed to reserve the
correspondence information while fusing intra-scale and aggregating cross-scale
stereo features. Our proposed TS3D achieves a 41.29% Moderate Car detection
average precision on the KITTI test set and takes 88 ms to detect objects from
each binocular image pair. It is competitive with advanced counterparts in
terms of both precision and inference speed.

本文提出 TS3D，一种基于 Transformer 的立体感知 3D 物体检测器，其中包括一种新颖的视差位置编码模型（DAPE）和 Stereo Reserving Feature Pyramid Network（SRFPN），使其在 KITTI 测试集上取得了 41.29％的平均精度。