Humans are in constant contact with the world as they move through it and
interact with it. This contact is a vital source of information for
understanding 3D humans, 3D scenes, and the interactions between them. In fact,
we demonstrate that these human-scene interactions (HSIs) can be leveraged to
improve the 3D reconstruction of a scene from a monocular RGB video. Our key
idea is that, as a person moves through a scene and interacts with it, we
accumulate HSIs across multiple input images, and optimize the 3D scene to
reconstruct a consistent, physically plausible and functional 3D scene layout.
Our optimization-based approach exploits three types of HSI constraints: (1)
humans that move in a scene are occluded or occlude objects, thus, defining the
depth ordering of the objects, (2) humans move through free space and do not
interpenetrate objects, (3) when humans and objects are in contact, the contact
surfaces occupy the same place in space. Using these constraints in an
optimization formulation across all observations, we significantly improve the
3D scene layout reconstruction. Furthermore, we show that our scene
reconstruction can be used to refine the initial 3D human pose and shape (HPS)
estimation. We evaluate the 3D scene layout reconstruction and HPS estimation
qualitatively and quantitatively using the PROX and PiGraphs datasets. The code
and data are available for research purposes at this https URL.

通过利用人与场景的交互可有效提高单目视频的 3D 场景重建，同时优化人的姿态估计，本论文基于三种 HSI 约束条件，即深度排序、是否穿插目标、接触面，使用优化方法再形成一致、物理合理且功能性 3D 场景布局，并对其在 PROX 和 PiGraphs 数据集上进行定量定性评估。

针对视觉环境重建的人类感知物体放置

Human-Aware Object Placement for Visual Environment Reconstruction

In this paper we tackle the problem of instance-level segmentation and depth
ordering from a single monocular image. Towards this goal, we take advantage of
convolutional neural nets and train them to directly predict instance-level
segmentations where the instance ID encodes the depth ordering within image
patches. To provide a coherent single explanation of an image we develop a
Markov random field which takes as input the predictions of convolutional
neural nets applied at overlapping patches of different resolutions, as well as
the output of a connected component algorithm. It aims to predict accurate
instance-level segmentation and depth ordering. We demonstrate the
effectiveness of our approach on the challenging KITTI benchmark and show good
performance on both tasks.

本文针对从单目图像进行实例级别分割和深度排序的问题，利用卷积神经网络训练来直接预测实例级别分割，同时采用马尔科夫随机场来提供图像的连贯单个解释，旨在预测准确的实例级别分割和深度排序，并在 KITTI 基准测试中表现出很好的性能。