Accurate human trajectory prediction is crucial for applications such as
autonomous vehicles, robotics, and surveillance systems. Yet, existing models
often fail to fully leverage the non-verbal social cues human subconsciously
communicate when navigating the space. To address this, we introduce
Social-Transmotion, a generic model that exploits the power of transformers to
handle diverse and numerous visual cues, capturing the multi-modal nature of
human behavior. We translate the idea of a prompt from Natural Language
Processing (NLP) to the task of human trajectory prediction, where a prompt can
be a sequence of x-y coordinates on the ground, bounding boxes or body poses.
This, in turn, augments trajectory data, leading to enhanced human trajectory
prediction. Our model exhibits flexibility and adaptability by capturing
spatiotemporal interactions between pedestrians based on the available visual
cues, whether they are poses, bounding boxes, or a combination thereof. By the
masking technique, we ensure our model's effectiveness even when certain visual
cues are unavailable, although performance is further boosted with the presence
of comprehensive visual data. We delve into the merits of using 2d versus 3d
poses, and a limited set of poses. Additionally, we investigate the spatial and
temporal attention map to identify which keypoints and frames of poses are
vital for optimizing human trajectory prediction. Our approach is validated on
multiple datasets, including JTA, JRDB, Pedestrians and Cyclists in Road
Traffic, and ETH-UCY. The code is publicly available:
this https URL

通过引入 Social-Transmotion，我们介绍了一种利用 transformer 处理多样化和大量视觉线索、捕捉人类行为多模态特性的通用模型，在人类轨迹预测任务中翻译自然语言处理中 prompt 的思想并扩充了轨迹数据，从而实现了增强的人类轨迹预测效果。