Convenient 4D modeling of human-object interactions is essential for numerous
applications. However, monocular tracking and rendering of complex interaction
scenarios remain challenging. In this paper, we propose Instant-NVR, a neural
approach for instant volumetric human-object tracking and rendering using a
single RGBD camera. It bridges traditional non-rigid tracking with recent
instant radiance field techniques via a multi-thread tracking-rendering
mechanism. In the tracking front-end, we adopt a robust human-object capture
scheme to provide sufficient motion priors. We further introduce a separated
instant neural representation with a novel hybrid deformation module for the
interacting scene. We also provide an on-the-fly reconstruction scheme of the
dynamic/static radiance fields via efficient motion-prior searching. Moreover,
we introduce an online key frame selection scheme and a rendering-aware
refinement strategy to significantly improve the appearance details for online
novel-view synthesis. Extensive experiments demonstrate the effectiveness and
efficiency of our approach for the instant generation of human-object radiance
fields on the fly, notably achieving real-time photo-realistic novel view
synthesis under complex human-object interactions.

本文提出了 Instant-NVR，一种利用单个 RGBD 相机进行即时容积式人物 - 物体跟踪和渲染的神经方法。该方法在采用高鲁棒性的捕获方案提供足够运动先验的同时，通过多线程跟踪 - 渲染机制，将传统的非刚性跟踪与最近的实时辐射场技术联系起来。本文还介绍了一个在线关键帧选择方案和一个渲染感知的细化策略，以显著提高在线新视角综合的外观细节。