Expressive human pose and shape estimation (a.k.a. 3D whole-body mesh
recovery) involves the human body, hand, and expression estimation. Most
existing methods have tackled this task in a two-stage manner, first detecting
the human body part with an off-the-shelf detection model and inferring the
different human body parts individually. Despite the impressive results
achieved, these methods suffer from 1) loss of valuable contextual information
via cropping, 2) introducing distractions, and 3) lacking inter-association
among different persons and body parts, inevitably causing performance
degradation, especially for crowded scenes. To address these issues, we
introduce a novel all-in-one-stage framework, AiOS, for multiple expressive
human pose and shape recovery without an additional human detection step.
Specifically, our method is built upon DETR, which treats multi-person
whole-body mesh recovery task as a progressive set prediction problem with
various sequential detection. We devise the decoder tokens and extend them to
our task. Specifically, we first employ a human token to probe a human location
in the image and encode global features for each instance, which provides a
coarse location for the later transformer block. Then, we introduce a
joint-related token to probe the human joint in the image and encoder a
fine-grained local feature, which collaborates with the global feature to
regress the whole-body mesh. This straightforward but effective model
outperforms previous state-of-the-art methods by a 9% reduction in NMVE on
AGORA, a 30% reduction in PVE on EHF, a 10% reduction in PVE on ARCTIC, and a
3% reduction in PVE on EgoBody.

提出了一种新的一体化框架 AiOS，用于多种表达性人体姿势和形状恢复，无需额外的人体检测步骤，通过 DETR 处理多人全身网格恢复任务，将其作为渐进集预测问题，通过解码器令牌和扩展实现对整个人体网格的回归，在 AGORA 上，NMVE 的减少率为 9%，EHF 上的 PVE 减少率为 30%，ARCTIC 上的 PVE 减少率为 10%，EgoBody 上的 PVE 减少率为 3%。

AiOS: 一阶段综合人体姿势和形状估计

AiOS: All-in-One-Stage Expressive Human Pose and Shape Estimation

Expressive human pose and shape estimation (EHPS) unifies body, hands, and
face motion capture with numerous applications. Despite encouraging progress,
current state-of-the-art methods still depend largely on confined training
datasets. In this work, we investigate scaling up EHPS towards the first
generalist foundation model (dubbed SMPLer-X), with up to ViT-Huge as the
backbone and training with up to 4.5M instances from diverse data sources. With
big data and the large model, SMPLer-X exhibits strong performance across
diverse test benchmarks and excellent transferability to even unseen
environments. 1) For the data scaling, we perform a systematic investigation on
32 EHPS datasets, encompassing a wide range of scenarios that a model trained
on any single dataset cannot handle. More importantly, capitalizing on insights
obtained from the extensive benchmarking process, we optimize our training
scheme and select datasets that lead to a significant leap in EHPS
capabilities. 2) For the model scaling, we take advantage of vision
transformers to study the scaling law of model sizes in EHPS. Moreover, our
finetuning strategy turn SMPLer-X into specialist models, allowing them to
achieve further performance boosts. Notably, our foundation model SMPLer-X
consistently delivers state-of-the-art results on seven benchmarks such as
AGORA (107.2 mm NMVE), UBody (57.4 mm PVE), EgoBody (63.6 mm PVE), and EHF
(62.3 mm PVE without finetuning).

通过大数据和大模型，本研究扩展了表达性人体姿势和形状估计 (EHPS) 模型，命名为 SMPLer-X，采用 ViT-Huge 作为骨干，并使用来自不同数据源的 450 万个实例进行训练。SMPLer-X 在多个测试基准上展现出强大性能，并且对于未知环境具有极高的可迁移性。