In recent times, there has been a growing interest in developing effective
perception techniques for combining information from multiple modalities. This
involves aligning features obtained from diverse sources to enable more
efficient training with larger datasets and constraints, as well as leveraging
the wealth of information contained in each modality. 2D and 3D Human Pose
Estimation (HPE) are two critical perceptual tasks in computer vision, which
have numerous downstream applications, such as Action Recognition,
Human-Computer Interaction, Object tracking, etc. Yet, there are limited
instances where the correlation between Image and 2D/3D human pose has been
clearly researched using a contrastive paradigm. In this paper, we propose
UniHPE, a unified Human Pose Estimation pipeline, which aligns features from
all three modalities, i.e., 2D human pose estimation, lifting-based and
image-based 3D human pose estimation, in the same pipeline. To align more than
two modalities at the same time, we propose a novel singular value based
contrastive learning loss, which better aligns different modalities and further
boosts the performance. In our evaluation, UniHPE achieves remarkable
performance metrics: MPJPE $50.5$mm on the Human3.6M dataset and PAMPJPE
$51.6$mm on the 3DPW dataset. Our proposed method holds immense potential to
advance the field of computer vision and contribute to various applications.

本文提出了 UniHPE，它是一个统一的人体姿势估计流程，通过同一流程中对所有三种模态进行特征对齐，即 2D 人体姿势估计、基于 lifting 和基于图像的 3D 人体姿势估计。我们提出了一种新的基于奇异值的对比学习损失函数来同时对齐多个模态，从而提高性能。评估结果显示 UniHPE 在 Human3.6M 数据集上的 MPJPE 为 50.5mm，3DPW 数据集上的 PAMPJPE 为 51.6mm，显示出巨大的潜力来推动计算机视觉领域并贡献于各种应用。

UniHPE：通过对比学习实现统一的人体姿势估计

UniHPE: Towards Unified Human Pose Estimation via Contrastive Learning

Autonomous driving perception techniques are typically based on supervised
machine learning models that are trained on real-world street data. A typical
training process involves capturing images with a single car model and
windshield configuration. However, deploying these trained models on different
car types can lead to a domain shift, which can potentially hurt the neural
networks performance and violate working ADAS requirements. To address this
issue, this paper investigates the domain shift problem further by evaluating
the sensitivity of two perception models to different windshield
configurations. This is done by evaluating the dependencies between neural
network benchmark metrics and optical merit functions by applying a Fourier
optics based threat model. Our results show that there is a performance gap
introduced by windshields and existing optical metrics used for posing
requirements might not be sufficient.

自动驾驶感知技术基于实际街道数据训练的监督式机器学习模型，然而，在不同车辆类型上部署这些训练模型可能会引起域漂移问题，对神经网络性能造成潜在影响并违反工作中的 ADAS 要求。本文通过评估两个感知模型对不同车窗配置的敏感性来深入研究领域漂移问题，使用傅立叶光学威胁模型应用神经网络基准指标与光学价值函数之间的依赖关系进行评估。结果表明车窗引入了性能差异，而现有的光学指标可能不足以提出要求。