Human pose and shape (HPS) estimation with lensless imaging is not only
beneficial to privacy protection but also can be used in covert surveillance
scenarios due to the small size and simple structure of this device. However,
this task presents significant challenges due to the inherent ambiguity of the
captured measurements and lacks effective methods for directly estimating human
pose and shape from lensless data. In this paper, we propose the first
end-to-end framework to recover 3D human poses and shapes from lensless
measurements to our knowledge. We specifically design a multi-scale lensless
feature decoder to decode the lensless measurements through the optically
encoded mask for efficient feature extraction. We also propose a double-head
auxiliary supervision mechanism to improve the estimation accuracy of human
limb ends. Besides, we establish a lensless imaging system and verify the
effectiveness of our method on various datasets acquired by our lensless
imaging system.

通过无镜成像，提出了第一个端到端的框架，从无镜成像数据中恢复三维人体姿势和形状。

LPSNet：无镜成像的端到端人体姿态和形状估计

LPSNet: End-to-End Human Pose and Shape Estimation with Lensless Imaging

Human Pose and Shape Estimation (HPSE) from RGB images can be broadly
categorized into two main groups: parametric and non-parametric approaches.
Parametric techniques leverage a low-dimensional statistical body model for
realistic results, whereas recent non-parametric methods achieve higher
precision by directly regressing the 3D coordinates of the human body. Despite
their strengths, both approaches face limitations: the parameters of
statistical body models pose challenges as regression targets, and predicting
3D coordinates introduces computational complexities and issues related to
smoothness. In this work, we take a novel approach to address the HPSE problem.
We introduce a unique method involving a low-dimensional discrete latent
representation of the human mesh, framing HPSE as a classification task.
Instead of predicting body model parameters or 3D vertex coordinates, our focus
is on forecasting the proposed discrete latent representation, which can be
decoded into a registered human mesh. This innovative paradigm offers two key
advantages: firstly, predicting a low-dimensional discrete representation
confines our predictions to the space of anthropomorphic poses and shapes;
secondly, by framing the problem as a classification task, we can harness the
discriminative power inherent in neural networks. Our proposed model, VQ-HPS, a
transformer-based architecture, forecasts the discrete latent representation of
the mesh, trained through minimizing a cross-entropy loss. Our results
demonstrate that VQ-HPS outperforms the current state-of-the-art non-parametric
approaches while yielding results as realistic as those produced by parametric
methods. This highlights the significant potential of the classification
approach for HPSE.

通过引入一种低维离散潜在表示的独特方法，我们将人体姿态和形状估计 (Human Pose and Shape Estimation，HPSE) 问题作为一个分类任务来解决，预测离散潜在表示，从而实现对人体网格的编码。我们的模型 VQ-HPS 在训练中通过最小化交叉熵损失实现，结果显示它在 HPSE 问题上胜过当前最先进的无参数方法，并且产生与参数方法一样逼真的结果，这突显了分类方法在 HPSE 中的重要潜力。

VQ-HPS：在向量量化潜变空间中的人体姿态和形状估计

VQ-HPS: Human Pose and Shape Estimation in a Vector-Quantized Latent  Space

Human pose and shape estimation (HPS) has attracted increasing attention in
recent years. While most existing studies focus on HPS from 2D images or videos
with inherent depth ambiguity, there are surging need to investigate HPS from
3D point clouds as depth sensors have been frequently employed in commercial
devices. However, real-world sensory 3D points are usually noisy and
incomplete, and also human bodies could have different poses of high diversity.
To tackle these challenges, we propose a principled framework, PointHPS, for
accurate 3D HPS from point clouds captured in real-world settings, which
iteratively refines point features through a cascaded architecture.
Specifically, each stage of PointHPS performs a series of downsampling and
upsampling operations to extract and collate both local and global cues, which
are further enhanced by two novel modules: 1) Cross-stage Feature Fusion (CFF)
for multi-scale feature propagation that allows information to flow effectively
through the stages, and 2) Intermediate Feature Enhancement (IFE) for
body-aware feature aggregation that improves feature quality after each stage.
To facilitate a comprehensive study under various scenarios, we conduct our
experiments on two large-scale benchmarks, comprising i) a dataset that
features diverse subjects and actions captured by real commercial sensors in a
laboratory environment, and ii) controlled synthetic data generated with
realistic considerations such as clothed humans in crowded outdoor scenes.
Extensive experiments demonstrate that PointHPS, with its powerful point
feature extraction and processing scheme, outperforms State-of-the-Art methods
by significant margins across the board. Homepage:
this https URL

基于 3D 点云数据的 PointHPS 框架通过迭代的特征提取和处理方案，在人体姿势和形状估计的研究领域中具有显著优势。

PointHPS: 基于点云的级联三维人体姿态和形状估计

PointHPS: Cascaded 3D Human Pose and Shape Estimation from Point Clouds

Human pose and shape (HPS) estimation methods achieve remarkable results.
However, current HPS benchmarks are mostly designed to test models in scenarios
that are similar to the training data. This can lead to critical situations in
real-world applications when the observed data differs significantly from the
training data and hence is out-of-distribution (OOD). It is therefore important
to test and improve the OOD robustness of HPS methods. To address this
fundamental problem, we develop a simulator that can be controlled in a
fine-grained manner using interpretable parameters to explore the manifold of
images of human pose, e.g. by varying poses, shapes, and clothes. We introduce
a learning-based testing method, termed PoseExaminer, that automatically
diagnoses HPS algorithms by searching over the parameter space of human pose
images to find the failure modes. Our strategy for exploring this
high-dimensional parameter space is a multi-agent reinforcement learning
system, in which the agents collaborate to explore different parts of the
parameter space. We show that our PoseExaminer discovers a variety of
limitations in current state-of-the-art models that are relevant in real-world
scenarios but are missed by current benchmarks. For example, it finds large
regions of realistic human poses that are not predicted correctly, as well as
reduced performance for humans with skinny and corpulent body shapes. In
addition, we show that fine-tuning HPS methods by exploiting the failure modes
found by PoseExaminer improve their robustness and even their performance on
standard benchmarks by a significant margin. The code are available for
research purposes.

为解决估计人体姿态和形状方法在真实场景中可能面临的数据偏移问题，本文提出了一种基于解释性参数的模拟器和一种学习测试方法 PoseExaminer，该方法利用多智能体强化学习系统协作探索高维人体姿态图像参数空间，通过揭示现有模型的局限性，并利用 PoseExaminer 发现的失败模式来进行微调，提高现有方法的鲁棒性和性能。