Recent transformer-based solutions have been introduced to estimate 3D human
pose from 2D keypoint sequence by considering body joints among all frames
globally to learn spatio-temporal correlation. We observe that the motions of
different joints differ significantly. However, the previous methods cannot
efficiently model the solid inter-frame correspondence of each joint, leading
to insufficient learning of spatial-temporal correlation. We propose MixSTE
(Mixed Spatio-Temporal Encoder), which has a temporal transformer block to
separately model the temporal motion of each joint and a spatial transformer
block to learn inter-joint spatial correlation. These two blocks are utilized
alternately to obtain better spatio-temporal feature encoding. In addition, the
network output is extended from the central frame to entire frames of the input
video, thereby improving the coherence between the input and output sequences.
Extensive experiments are conducted on three benchmarks (Human3.6M,
MPI-INF-3DHP, and HumanEva). The results show that our model outperforms the
state-of-the-art approach by 10.9% P-MPJPE and 7.6% MPJPE. The code is
available at this https URL

本文提出了一种叫做 MixSTE 的混合的时空编码器，该编码器通过时间和空间变换块分别对每个关节的运动进行建模以及学习关节之间的空间相关性，可在 3D 人体姿态估计任务上取得更好的性能表现。