We introduce HOSNeRF, a novel 360{\deg} free-viewpoint rendering method that
reconstructs neural radiance fields for dynamic human-object-scene from a
single monocular in-the-wild video. Our method enables pausing the video at any
frame and rendering all scene details (dynamic humans, objects, and
backgrounds) from arbitrary viewpoints. The first challenge in this task is the
complex object motions in human-object interactions, which we tackle by
introducing the new object bones into the conventional human skeleton hierarchy
to effectively estimate large object deformations in our dynamic human-object
model. The second challenge is that humans interact with different objects at
different times, for which we introduce two new learnable object state
embeddings that can be used as conditions for learning our human-object
representation and scene representation, respectively. Extensive experiments
show that HOSNeRF significantly outperforms SOTA approaches on two challenging
datasets by a large margin of 40% ~ 50% in terms of LPIPS. The code, data, and
compelling examples of 360{\deg} free-viewpoint renderings from single videos
will be released in this https URL

本论文提出 HOSNeRF 方法，通过将新的物体骨骼引入传统的人类骨骼层次结构，有效地估计动态人体 - 物体 - 场景的 large object deformations，并引入两种学习可用的目标状态嵌入，可以用于学习人体 - 物体表示和场景表示。通过广泛实验，表明 HOSNeRF 在两个具有挑战性的数据集上的表现显著优于最先进的方法。