Rendering dynamic 3D human from monocular videos is crucial for various
applications such as virtual reality and digital entertainment. Most methods
assume the people is in an unobstructed scene, while various objects may cause
the occlusion of body parts in real-life scenarios. Previous method utilizing
NeRF for surface rendering to recover the occluded areas, but it requiring more
than one day to train and several seconds to render, failing to meet the
requirements of real-time interactive applications. To address these issues, we
propose OccGaussian based on 3D Gaussian Splatting, which can be trained within
6 minutes and produces high-quality human renderings up to 160 FPS with
occluded input. OccGaussian initializes 3D Gaussian distributions in the
canonical space, and we perform occlusion feature query at occluded regions,
the aggregated pixel-align feature is extracted to compensate for the missing
information. Then we use Gaussian Feature MLP to further process the feature
along with the occlusion-aware loss functions to better perceive the occluded
area. Extensive experiments both in simulated and real-world occlusions,
demonstrate that our method achieves comparable or even superior performance
compared to the state-of-the-art method. And we improving training and
inference speeds by 250x and 800x, respectively. Our code will be available for
research purposes.

我们提出了基于 3D 高斯散射的 OccGaussian 方法，其可在 6 分钟内进行训练，并且以高质量 160FPS 渲染具有遮挡输入的人体场景。通过在标准空间中初始化 3D 高斯分布，并在遮挡区域进行遮挡特征查询，提取聚合的像素对齐特征来弥补丢失的信息，然后我们使用高斯特征 MLP 进一步处理特征，并结合面向遮挡区域的损失函数来更好地感知遮挡区域。大量的实验结果表明，我们的方法在模拟和真实世界遮挡场景中实现了与最先进方法相媲美甚至更优的性能，且训练和推理速度提高了 250 倍和 800 倍，我们的代码将提供给研究目的使用。

OccGaussian：用于遮挡人体渲染的三维高斯粒子

OccGaussian: 3D Gaussian Splatting for Occluded Human Rendering

We propose Medial Atom Ray Fields (MARFs), a novel neural object
representation that enables accurate differentiable surface rendering with a
single network evaluation per camera ray. Existing neural ray fields struggle
with multi-view consistency and representing surface discontinuities. MARFs
address both using a medial shape representation, a dual representation of
solid geometry that yields cheap geometrically grounded surface normals, in
turn enabling computing analytical curvature despite the network having no
second derivative. MARFs map a camera ray to multiple medial intersection
candidates, subject to ray-sphere intersection testing. We illustrate how the
learned medial shape quantities applies to sub-surface scattering, part
segmentation, and aid representing a space of articulated shapes. Able to learn
a space of shape priors, MARFs may prove useful for tasks like shape retrieval
and shape completion, among others. Code and data can be found at
this https URL

本文提出了一种新的神经物体表示方法 ——MEDIAL ATOM RAY FIELDS (MARFs)，使用单个网络评估实现精确可微分表面渲染，MARFs 地址了使用介质形状表示法的多视图一致性和表面间断的问题，并提供了便宜的地理基准表面法线，MARFs 映射一个相机光线到多个介质交点候选者，这可能会对形状检索和形状完成等任务有用。

MARF：中心原子射线场物体表示法

MARF: The Medial Atom Ray Field Object Representation

We propose a novel method that renders point clouds as if they are surfaces.
The proposed method is differentiable and requires no scene-specific
optimization. This unique capability enables, out-of-the-box, surface normal
estimation, rendering room-scale point clouds, inverse rendering, and ray
tracing with global illumination. Unlike existing work that focuses on
converting point clouds to other representations--e.g., surfaces or implicit
functions--our key idea is to directly infer the intersection of a light ray
with the underlying surface represented by the given point cloud. Specifically,
we train a set transformer that, given a small number of local neighbor points
along a light ray, provides the intersection point, the surface normal, and the
material blending weights, which are used to render the outcome of this light
ray. Localizing the problem into small neighborhoods enables us to train a
model with only 48 meshes and apply it to unseen point clouds. Our model
achieves higher estimation accuracy than state-of-the-art surface
reconstruction and point-cloud rendering methods on three test sets. When
applied to room-scale point clouds, without any scene-specific optimization,
the model achieves competitive quality with the state-of-the-art novel-view
rendering methods. Moreover, we demonstrate ability to render and manipulate
Lidar-scanned point clouds such as lighting control and object insertion.

本文提出了一种新的方法，将点云渲染成表面，该方法可微并且不需要特定于场景的优化，可以用于表面法线估计、渲染全局点云、反向渲染和光线跟踪。通过直接推断给定点云所代表的底层表面与光线的交点，通过训练一个集合转换器可以将局部邻居点与光线的交点、表面法线和材料混合权重结合起来，使用 48 个网格进行模型训练，可应用于未见过的点云上，实现了比现有表面重建和点云渲染方法更高的估计精度。