We propose a novel Transformer-based architecture for the task of generative
modelling of 3D human motion. Previous work commonly relies on RNN-based models
considering shorter forecast horizons reaching a stationary and often
implausible state quickly. Recent studies show that implicit temporal
representations in the frequency domain are also effective in making
predictions for a predetermined horizon. Our focus lies on learning
spatio-temporal representations autoregressively and hence generation of
plausible future developments over both short and long term. The proposed model
learns high dimensional embeddings for skeletal joints and how to compose a
temporally coherent pose via a decoupled temporal and spatial self-attention
mechanism. Our dual attention concept allows the model to access current and
past information directly and to capture both the structural and the temporal
dependencies explicitly. We show empirically that this effectively learns the
underlying motion dynamics and reduces error accumulation over time observed in
auto-regressive models. Our model is able to make accurate short-term
predictions and generate plausible motion sequences over long horizons. We make
our code publicly available at this https URL

本文提出一种新颖的基于 Transformer 的架构，用于生成 3D 人体运动的模型。该模型能够学习高维度骨骼关节的嵌入，通过解耦的时间和空间自注意机制使得生成的姿态在时间上是连贯的。实验结果表明该模型能够准确地生成短期预测并在长期时间跨度内生成合理的运动序列。

用于三维人体运动预测的时空转换器

A Spatio-temporal Transformer for 3D Human Motion Prediction

In this paper, mm-Pose, a novel approach to detect and track human skeletons
in real-time using an mmWave radar, is proposed. To the best of the authors'
knowledge, this is the first method to detect >15 distinct skeletal joints
using mmWave radar reflection signals. The proposed method would find several
applications in traffic monitoring systems, autonomous vehicles, patient
monitoring systems and defense forces to detect and track human skeleton for
effective and preventive decision making in real-time. The use of radar makes
the system operationally robust to scene lighting and adverse weather
conditions. The reflected radar point cloud in range, azimuth and elevation are
first resolved and projected in Range-Azimuth and Range-Elevation planes. A
novel low-size high-resolution radar-to-image representation is also presented,
that overcomes the sparsity in traditional point cloud data and offers
significant reduction in the subsequent machine learning architecture. The RGB
channels were assigned with the normalized values of range, elevation/azimuth
and the power level of the reflection signals for each of the points. A forked
CNN architecture was used to predict the real-world position of the skeletal
joints in 3-D space, using the radar-to-image representation. The proposed
method was tested for a single human scenario for four primary motions, (i)
Walking, (ii) Swinging left arm, (iii) Swinging right arm, and (iv) Swinging
both arms to validate accurate predictions for motion in range, azimuth and
elevation. The detailed methodology, implementation, challenges, and validation
results are presented.

该研究提出了一种新的使用 mmWave 雷达在实时情况下检测和跟踪人体骨架的方法，使用机器学习进行骨骼关节的三维定位，可应用于交通监控、自动驾驶、病人监测和国防军事等领域，并且克服了光照和恶劣气象条件的影响。