With wearable IMU sensors, one can estimate human poses from wearable devices without requiring visual input \cite{von2017sparse}. In this work, we pose the question: Can we reason about object structure in real-world environments solely from human trajectory information? Crucially, we observe that human motion and interactions tend to give strong information about the objects in a scene -- for instance a person sitting indicates the likely presence of a chair or sofa. To this end, we propose P2R-Net to learn a probabilistic 3D model of the objects in a scene characterized by their class categories and oriented 3D bounding boxes, based on an input observed human trajectory in the environment. P2R-Net models the probability distribution of object class as well as a deep Gaussian mixture model for object boxes, enabling sampling of multiple, diverse, likely modes of object configurations from an observed human trajectory. In our experiments we demonstrate that P2R-Net can effectively learn multi-modal distributions of likely objects for human motions, and produce a variety of plausible object structures of the environment, even without any visual information.

本研究使用穿戴式IMU传感器，基于人的运动轨迹在真实环境中构建概率三维模型，通过P2R-Net模型扩展多模态的物体配置，不需要视觉信息即可准确地评估人物结构和场景中的物品结构，并在PROX数据集和VirtualHome平台上表现优异。

Pose2Room：从人类活动中理解 3D 场景