We describe a method for 3D human pose estimation from transient images
(i.e., a 3D spatio-temporal histogram of photons) acquired by an optical
non-line-of-sight (NLOS) imaging system. Our method can perceive 3D human pose
by `looking around corners' through the use of light indirectly reflected by
the environment. We bring together a diverse set of technologies from NLOS
imaging, human pose estimation and deep reinforcement learning to construct an
end-to-end data processing pipeline that converts a raw stream of photon
measurements into a full 3D human pose sequence estimate. Our contributions are
the design of data representation process which includes (1) a learnable
inverse point spread function (PSF) to convert raw transient images into a deep
feature vector; (2) a neural humanoid control policy conditioned on the
transient image feature and learned from interactions with a physics simulator;
and (3) a data synthesis and augmentation strategy based on depth data that can
be transferred to a real-world NLOS imaging system. Our preliminary experiments
suggest that our method is able to generalize to real-world NLOS measurement to
estimate physically-valid 3D human poses.

本文通过光学非视线成像系统获取的瞬态图像描述了一种实现 3D 人体姿势估计的方法，通过使用间接反射的光线，实现了从街角窥视的功能；该方法将非视线成像、人体姿势估计和深度强化学习等多个领域的技术结合起来，通过端对端的数据处理流程，将源源不断的光子测量转换为整个 3D 人体姿势序列的估计，并通过数据综合和增强策略实现靠近现实的 NLOS 成像系统数据的预测。初步实验结果表明，本文所提出的方法可以推广至真实世界的 NLOS 测量，从而实现估算物理上有效的 3D 人体姿势。