Multi-person motion prediction is a challenging problem due to the dependency
of motion on both individual past movements and interactions with other people.
Transformer-based methods have shown promising results on this task, but they
miss the explicit relation representation between joints, such as skeleton
structure and pairwise distance, which is crucial for accurate interaction
modeling. In this paper, we propose the Joint-Relation Transformer, which
utilizes relation information to enhance interaction modeling and improve
future motion prediction. Our relation information contains the relative
distance and the intra-/inter-person physical constraints. To fuse relation and
joint information, we design a novel joint-relation fusion layer with
relation-aware attention to update both features. Additionally, we supervise
the relation information by forecasting future distance. Experiments show that
our method achieves a 13.4% improvement of 900ms VIM on 3DPW-SoMoF/RC and
17.8%/12.0% improvement of 3s MPJPE on CMU-Mpcap/MuPoTS-3D dataset.

我们提出了关节 - 关系转换器，利用关系信息来增强交互建模和提高未来运动预测的准确性。

多人动作预测的联合关系变换器

Joint-Relation Transformer for Multi-Person Motion Prediction

Humans exhibit complex motions that vary depending on the task that they are
performing, the interactions they engage in, as well as subject-specific
preferences. Therefore, forecasting future poses based on the history of the
previous motions is a challenging task. This paper presents an innovative
auxiliary-memory-powered deep neural network framework for the improved
modelling of historical knowledge. Specifically, we disentangle
subject-specific, task-specific, and other auxiliary information from the
observed pose sequences and utilise these factorised features to query the
memory. A novel Multi-Head knowledge retrieval scheme leverages these
factorised feature embeddings to perform multiple querying operations over the
historical observations captured within the auxiliary memory. Moreover, our
proposed dynamic masking strategy makes this feature disentanglement process
dynamic. Two novel loss functions are introduced to encourage diversity within
the auxiliary memory while ensuring the stability of the memory contents, such
that it can locate and store salient information that can aid the long-term
prediction of future motion, irrespective of data imbalances or the diversity
of the input data distribution. With extensive experiments conducted on two
public benchmarks, Human3.6M and CMU-Mocap, we demonstrate that these design
choices collectively allow the proposed approach to outperform the current
state-of-the-art methods by significant margins: $>$ 17\% on the Human3.6M
dataset and $>$ 9\% on the CMU-Mocap dataset.

本文提出了一种用于预测未来动作的深度神经网络框架，其中使用辅助记忆动态地导出主体特定的、任务特定的和其他辅助信息，并提出一个多头知识检索机制，以允许进行多个查询操作以促进将来动作的长期预测。在 Human3.6M 和 CMU-Mocap 两个公共基准测试中，我们证明了这些设计选择允许所提出的方法以显着优于当前最先进方法的较大优势进行操作。

记住重要的事情：基于分解多头检索和辅助记忆稳定方案的人体运动预测

Remembering What Is Important: A Factorised Multi-Head Retrieval and  Auxiliary Memory Stabilisation Scheme for Human Motion Prediction

In this paper, we address the problem of predicting the future motion of a
dynamic agent (called a target agent) given its current and past states as well
as the information on its environment. It is paramount to develop a prediction
model that can exploit the contextual information in both static and dynamic
environments surrounding the target agent and generate diverse trajectory
samples that are meaningful in a traffic context. We propose a novel prediction
model, referred to as the lane-aware prediction (LaPred) network, which uses
the instance-level lane entities extracted from a semantic map to predict the
multi-modal future trajectories. For each lane candidate found in the
neighborhood of the target agent, LaPred extracts the joint features relating
the lane and the trajectories of the neighboring agents. Then, the features for
all lane candidates are fused with the attention weights learned through a
self-supervised learning task that identifies the lane candidate likely to be
followed by the target agent. Using the instance-level lane information, LaPred
can produce the trajectories compliant with the surroundings better than 2D
raster image-based methods and generate the diverse future trajectories given
multiple lane candidates. The experiments conducted on the public nuScenes
dataset and Argoverse dataset demonstrate that the proposed LaPred method
significantly outperforms the existing prediction models, achieving
state-of-the-art performance in the benchmarks.

本文提出了一种新的预测模型 LaPred，利用语义地图中提取的车道实例来预测多模态的未来轨迹，通过自监督学习任务识别可能由目标车辆跟随的车道实例，并产生比基于 2D 光栅图像的方法更好地符合周围环境的轨迹，实验表明该模型在公共 nuScenes 数据集和 Argoverse 数据集上显著优于现有的预测模型，达到了基准测试的最新性能。