We present JRDB, a novel egocentric dataset collected from our social mobile
manipulator JackRabbot. The dataset includes 64 minutes of annotated multimodal
sensor data including stereo cylindrical 360$^\circ$ RGB video at 15 fps, 3D
point clouds from two Velodyne 16 Lidars, line 3D point clouds from two Sick
Lidars, audio signal, RGB-D video at 30 fps, 360$^\circ$ spherical image from a
fisheye camera and encoder values from the robot's wheels. Our dataset
incorporates data from traditionally underrepresented scenes such as indoor
environments and pedestrian areas, all from the ego-perspective of the robot,
both stationary and navigating. The dataset has been annotated with over 2.3
million bounding boxes spread over 5 individual cameras and 1.8 million
associated 3D cuboids around all people in the scenes totaling over 3500 time
consistent trajectories. Together with our dataset and the annotations, we
launch a benchmark and metrics for 2D and 3D person detection and tracking.
With this dataset, which we plan on extending with further types of annotation
in the future, we hope to provide a new source of data and a test-bench for
research in the areas of egocentric robot vision, autonomous navigation, and
all perceptual tasks around social robotics in human environments.

介绍了 JRDB 数据集，该数据集是从社交机器人 JackRabbot 采集的一系列包括 RGB 视频、点云、音频信号等多模态数据，并标记了超过 230 万个包围框及 3500 个时间连续轨迹，其可用于研究以机器人视角为主的视觉算法、人物探测和自主导航等领域。