Compared with traditional RGB-only visual tracking, few datasets have been
constructed for RGB-D tracking. In this paper, we propose ARKitTrack, a new
RGB-D tracking dataset for both static and dynamic scenes captured by
consumer-grade LiDAR scanners equipped on Apple's iPhone and iPad. ARKitTrack
contains 300 RGB-D sequences, 455 targets, and 229.7K video frames in total.
Along with the bounding box annotations and frame-level attributes, we also
annotate this dataset with 123.9K pixel-level target masks. Besides, the camera
intrinsic and camera pose of each frame are provided for future developments.
To demonstrate the potential usefulness of this dataset, we further present a
unified baseline for both box-level and pixel-level tracking, which integrates
RGB features with bird's-eye-view representations to better explore
cross-modality 3D geometry. In-depth empirical analysis has verified that the
ARKitTrack dataset can significantly facilitate RGB-D tracking and that the
proposed baseline method compares favorably against the state of the arts. The
code and dataset is available at this https URL

本论文提出 ARKitTrack 数据集，包含 300 个 RGB-D 序列，455 个目标，总计 229.7K 个视频帧和 123.9K 像素级目标遮罩，并提供每个帧的相机内参和相机姿态信息。此外，研究人员还提出了一种新的跨模态 3D 几何一般方法，该方法将 RGB 特征与俯视图表示相结合，以更好地探索交叉模态 3D 几何信息。实验证明，ARKitTrack 数据集可以显著促进 RGB-D 跟踪，并且所提出的基线方法比当前技术水平更具优势。