Humans live within a 3D space and constantly interact with it to perform
tasks. Such interactions involve physical contact between surfaces that is
semantically meaningful. Our goal is to learn how humans interact with scenes
and leverage this to enable virtual characters to do the same. To that end, we
introduce a novel Human-Scene Interaction (HSI) model that encodes proximal
relationships, called POSA for "Pose with prOximitieS and contActs". The
representation of interaction is body-centric, which enables it to generalize
to new scenes. Specifically, POSA augments the SMPL-X parametric human body
model such that, for every mesh vertex, it encodes (a) the contact probability
with the scene surface and (b) the corresponding semantic scene label. We learn
POSA with a VAE conditioned on the SMPL-X vertices, and train on the PROX
dataset, which contains SMPL-X meshes of people interacting with 3D scenes, and
the corresponding scene semantics from the PROX-E dataset. We demonstrate the
value of POSA with two applications. First, we automatically place 3D scans of
people in scenes. We use a SMPL-X model fit to the scan as a proxy and then
find its most likely placement in 3D. POSA provides an effective representation
to search for "affordances" in the scene that match the likely contact
relationships for that pose. We perform a perceptual study that shows
significant improvement over the state of the art on this task. Second, we show
that POSA's learned representation of body-scene interaction supports monocular
human pose estimation that is consistent with a 3D scene, improving on the
state of the art. Our model and code are available for research purposes at
this https URL

本文提出了 POSA 模型，用来学习人体与场景的交互，包括接触概率和语义场景标签，并展示了其在 3D 人物自动放置和一致的单目人体姿态估计方面的改进。