Human-centric perception (e.g. pedetrian detection, segmentation, pose
estimation, and attribute analysis) is a long-standing problem for computer
vision. This paper introduces a unified and versatile framework (HQNet) for
single-stage multi-person multi-task human-centric perception (HCP). Our
approach centers on learning a unified human query representation, denoted as
Human Query, which captures intricate instance-level features for individual
persons and disentangles complex multi-person scenarios. Although different HCP
tasks have been well-studied individually, single-stage multi-task learning of
HCP tasks has not been fully exploited in the literature due to the absence of
a comprehensive benchmark dataset. To address this gap, we propose
COCO-UniHuman benchmark dataset to enable model development and comprehensive
evaluation. Experimental results demonstrate the proposed method's
state-of-the-art performance among multi-task HCP models and its competitive
performance compared to task-specific HCP models. Moreover, our experiments
underscore Human Query's adaptability to new HCP tasks, thus demonstrating its
robust generalization capability. Codes and data will be publicly accessible.

该研究提出了一种统一而全面的人类中心感知 (HCP) 框架 (HQNet)，通过学习一个统一的人类查询表示 (Human Query) 来解决单阶段多人多任务的人类中心感知问题。实验结果表明该方法在多任务人类中心感知模型中表现出色，并与任务特定的人类中心感知模型的性能相媲美，同时还展示了人类查询在新的 HCP 任务上的适应性和鲁棒的泛化能力。

只学一次查询：单阶段多人多任务人类中心感知的统一人类查询学习

You Only Learn One Query: Learning Unified Human Query for Single-Stage  Multi-Person Multi-Task Human-Centric Perception

Human-centric perception plays a vital role in vision and graphics. But their
data annotations are prohibitively expensive. Therefore, it is desirable to
have a versatile pre-train model that serves as a foundation for data-efficient
downstream tasks transfer. To this end, we propose the Human-Centric
Multi-Modal Contrastive Learning framework HCMoCo that leverages the
multi-modal nature of human data (e.g. RGB, depth, 2D keypoints) for effective
representation learning. The objective comes with two main challenges: dense
pre-train for multi-modality data, efficient usage of sparse human priors. To
tackle the challenges, we design the novel Dense Intra-sample Contrastive
Learning and Sparse Structure-aware Contrastive Learning targets by
hierarchically learning a modal-invariant latent space featured with continuous
and ordinal feature distribution and structure-aware semantic consistency.
HCMoCo provides pre-train for different modalities by combining heterogeneous
datasets, which allows efficient usage of existing task-specific human data.
Extensive experiments on four downstream tasks of different modalities
demonstrate the effectiveness of HCMoCo, especially under data-efficient
settings (7.16% and 12% improvement on DensePose Estimation and Human Parsing).
Moreover, we demonstrate the versatility of HCMoCo by exploring cross-modality
supervision and missing-modality inference, validating its strong ability in
cross-modal association and reasoning.

本文提出 Human-Centric Multi-Modal Contrastive Learning 框架，通过密集内部样本对比学习和稀疏结构感知对比学习目标，实现多视角数据有效特征表示，为数据高效下游任务转移提供了通用的预训练模型。