In recent times, there has been a growing interest in developing effective
perception techniques for combining information from multiple modalities. This
involves aligning features obtained from diverse sources to enable more
efficient training with larger datasets and constraints, as well as leveraging
the wealth of information contained in each modality. 2D and 3D Human Pose
Estimation (HPE) are two critical perceptual tasks in computer vision, which
have numerous downstream applications, such as Action Recognition,
Human-Computer Interaction, Object tracking, etc. Yet, there are limited
instances where the correlation between Image and 2D/3D human pose has been
clearly researched using a contrastive paradigm. In this paper, we propose
UniHPE, a unified Human Pose Estimation pipeline, which aligns features from
all three modalities, i.e., 2D human pose estimation, lifting-based and
image-based 3D human pose estimation, in the same pipeline. To align more than
two modalities at the same time, we propose a novel singular value based
contrastive learning loss, which better aligns different modalities and further
boosts the performance. In our evaluation, UniHPE achieves remarkable
performance metrics: MPJPE $50.5$mm on the Human3.6M dataset and PAMPJPE
$51.6$mm on the 3DPW dataset. Our proposed method holds immense potential to
advance the field of computer vision and contribute to various applications.

本文提出了 UniHPE，它是一个统一的人体姿势估计流程，通过同一流程中对所有三种模态进行特征对齐，即 2D 人体姿势估计、基于 lifting 和基于图像的 3D 人体姿势估计。我们提出了一种新的基于奇异值的对比学习损失函数来同时对齐多个模态，从而提高性能。评估结果显示 UniHPE 在 Human3.6M 数据集上的 MPJPE 为 50.5mm，3DPW 数据集上的 PAMPJPE 为 51.6mm，显示出巨大的潜力来推动计算机视觉领域并贡献于各种应用。