Accurately detecting and tracking pedestrians in 3D space is challenging due
to large variations in rotations, poses and scales. The situation becomes even
worse for dense crowds with severe occlusions. However, existing benchmarks
either only provide 2D annotations, or have limited 3D annotations with
low-density pedestrian distribution, making it difficult to build a reliable
pedestrian perception system especially in crowded scenes. To better evaluate
pedestrian perception algorithms in crowded scenarios, we introduce a
large-scale multimodal dataset,STCrowd. Specifically, in STCrowd, there are a
total of 219 K pedestrian instances and 20 persons per frame on average, with
various levels of occlusion. We provide synchronized LiDAR point clouds and
camera images as well as their corresponding 3D labels and joint IDs. STCrowd
can be used for various tasks, including LiDAR-only, image-only, and
sensor-fusion based pedestrian detection and tracking. We provide baselines for
most of the tasks. In addition, considering the property of sparse global
distribution and density-varying local distribution of pedestrians, we further
propose a novel method, Density-aware Hierarchical heatmap Aggregation (DHA),
to enhance pedestrian perception in crowded scenes. Extensive experiments show
that our new method achieves state-of-the-art performance for pedestrian
detection on various datasets.

为了更好地评估拥挤场景下的行人感知算法，我们引入了一个大规模多模态数据集 STCrowd，并提出了一种新的方法 DHA 来增强拥挤场景中的行人感知。实验表明，我们的方法在各种数据集上均取得了最先进的性能。