Rendering the visual appearance of moving humans from occluded monocular
videos is a challenging task. Most existing research renders 3D humans under
ideal conditions, requiring a clear and unobstructed scene. Those methods
cannot be used to render humans in real-world scenes where obstacles may block
the camera's view and lead to partial occlusions. In this work, we present
Wild2Avatar, a neural rendering approach catered for occluded in-the-wild
monocular videos. We propose occlusion-aware scene parameterization for
decoupling the scene into three parts - occlusion, human, and background.
Additionally, extensive objective functions are designed to help enforce the
decoupling of the human from both the occlusion and the background and to
ensure the completeness of the human model. We verify the effectiveness of our
approach with experiments on in-the-wild videos.

通过 Wild2Avatar，在遮挡的自然单目视频中呈现出移动人物的视觉外观，该方法通过将场景分解为遮挡、人物和背景三个部分，设计广泛的客观函数以帮助实现人物与遮挡和背景的解耦，并确保人体模型的完整性，通过在自然环境下进行实验证明了该方法的有效性。

Wild2Avatar: 遮挡背后的人类渲染

Wild2Avatar: Rendering Humans Behind Occlusions

3D understanding and rendering of moving humans from monocular videos is a
challenging task. Despite recent progress, the task remains difficult in
real-world scenarios, where obstacles may block the camera view and cause
partial occlusions in the captured videos. Existing methods cannot handle such
defects due to two reasons. First, the standard rendering strategy relies on
point-point mapping, which could lead to dramatic disparities between the
visible and occluded areas of the body. Second, the naive direct regression
approach does not consider any feasibility criteria (ie, prior information) for
rendering under occlusions. To tackle the above drawbacks, we present OccNeRF,
a neural rendering method that achieves better rendering of humans in severely
occluded scenes. As direct solutions to the two drawbacks, we propose
surface-based rendering by integrating geometry and visibility priors. We
validate our method on both simulated and real-world occlusions and demonstrate
our method's superiority.

我们提出了一种新的神经渲染方法 OccNeRF，通过整合几何和可见性先验，实现了在严重遮挡场景中更好地渲染人类形象。我们验证了该方法在模拟和真实遮挡情况下的优越性。

从物体遮挡的单目视频中渲染人类

Rendering Humans from Object-Occluded Monocular Videos

Although the estimation of 3D human pose and shape (HPS) is rapidly
progressing, current methods still cannot reliably estimate moving humans in
global coordinates, which is critical for many applications. This is
particularly challenging when the camera is also moving, entangling human and
camera motion. To address these issues, we adopt a novel 5D representation
(space, time, and identity) that enables end-to-end reasoning about people in
scenes. Our method, called TRACE, introduces several novel architectural
components. Most importantly, it uses two new "maps" to reason about the 3D
trajectory of people over time in camera, and world, coordinates. An additional
memory unit enables persistent tracking of people even during long occlusions.
TRACE is the first one-stage method to jointly recover and track 3D humans in
global coordinates from dynamic cameras. By training it end-to-end, and using
full image information, TRACE achieves state-of-the-art performance on tracking
and HPS benchmarks. The code and dataset are released for research purposes.

TRACE 是一种使用 5D 表示法并引入了两个新 “地图” 以推理关于动态摄像机下人物全局坐标中 3D 轨迹的方法，使用端到端模型训练，实现了在跟踪和 HPS 基准上的最先进性能。