Estimating 3D human poses from monocular videos is a challenging task due to
depth ambiguity and self-occlusion. Most existing works attempt to solve both
issues by exploiting spatial and temporal relationships. However, those works
ignore the fact that it is an inverse problem where multiple feasible solutions
(i.e., hypotheses) exist. To relieve this limitation, we propose a
Multi-Hypothesis Transformer (MHFormer) that learns spatio-temporal
representations of multiple plausible pose hypotheses. In order to effectively
model multi-hypothesis dependencies and build strong relationships across
hypothesis features, the task is decomposed into three stages: (i) Generate
multiple initial hypothesis representations; (ii) Model self-hypothesis
communication, merge multiple hypotheses into a single converged representation
and then partition it into several diverged hypotheses; (iii) Learn
cross-hypothesis communication and aggregate the multi-hypothesis features to
synthesize the final 3D pose. Through the above processes, the final
representation is enhanced and the synthesized pose is much more accurate.
Extensive experiments show that MHFormer achieves state-of-the-art results on
two challenging datasets: Human3.6M and MPI-INF-3DHP. Without bells and
whistles, its performance surpasses the previous best result by a large margin
of 3% on Human3.6M. Code and models are available at
https://github.com/Vegetebird/MHFormer.

本文提出了一种名为 Multi-Hypothesis Transformer (MHFormer) 的模型，旨在通过学习多个合理的姿势假设的空时表示，解决单目视频中的 3D 人体姿态估计问题，并在两个具有挑战性的数据集上取得了最新的最优结果。