Perceiving the world and forecasting its future state is a critical task for
self-driving. Supervised approaches leverage annotated object labels to learn a
model of the world -- traditionally with object detections and trajectory
predictions, or temporal bird's-eye-view (BEV) occupancy fields. However, these
annotations are expensive and typically limited to a set of predefined
categories that do not cover everything we might encounter on the road.
Instead, we learn to perceive and forecast a continuous 4D (spatio-temporal)
occupancy field with self-supervision from LiDAR data. This unsupervised world
model can be easily and effectively transferred to downstream tasks. We tackle
point cloud forecasting by adding a lightweight learned renderer and achieve
state-of-the-art performance in Argoverse 2, nuScenes, and KITTI. To further
showcase its transferability, we fine-tune our model for BEV semantic occupancy
forecasting and show that it outperforms the fully supervised state-of-the-art,
especially when labeled data is scarce. Finally, when compared to prior
state-of-the-art on spatio-temporal geometric occupancy prediction, our 4D
world model achieves a much higher recall of objects from classes relevant to
self-driving.

我们提出了一种使用自监督从 LiDAR 数据学习连续的 4D（时空）占据场的无监督世界模型，该模型可以轻松有效地迁移到下游任务，并在点云预测和 BEV 语义占据预测方面取得了最先进的性能，特别是在标注数据稀缺时。此外，在与先前的时空几何占据预测的最新技术进行比较时，我们的 4D 世界模型对于与自动驾驶相关的类别的对象的召回率要高得多。

UnO: 无监督占用场景的感知和预测

UnO: Unsupervised Occupancy Fields for Perception and Forecasting

A human 3D avatar is one of the important elements in the metaverse, and the
modeling effect directly affects people's visual experience. However, the human
body has a complex topology and diverse details, so it is often expensive,
time-consuming, and laborious to build a satisfactory model. Recent studies
have proposed a novel method, implicit neural representation, which is a
continuous representation method and can describe objects with arbitrary
topology at arbitrary resolution. Researchers have applied implicit neural
representation to human 3D avatar modeling and obtained more excellent results
than traditional methods. This paper comprehensively reviews the application of
implicit neural representation in human body modeling. First, we introduce
three implicit representations of occupancy field, SDF, and NeRF, and make a
classification of the literature investigated in this paper. Then the
application of implicit modeling methods in the body, hand, and head are
compared and analyzed respectively. Finally, we point out the shortcomings of
current work and provide available suggestions for researchers.

本文全面回顾了隐式神经表达在人体建模中的应用，比较分析了隐式建模方法在身体、手和头部的应用，并指出了当前工作的不足之处和研究人员的可用建议。

使用隐式神经表示进行人类 3D-avatar 建模：简要调查

Human 3D Avatar Modeling with Implicit Neural Representation: A Brief  Survey

Detailed 3D reconstruction and photo-realistic relighting of digital humans
are essential for various applications. To this end, we propose a novel
sparse-view 3d human reconstruction framework that closely incorporates the
occupancy field and albedo field with an additional visibility field--it not
only resolves occlusion ambiguity in multiview feature aggregation, but can
also be used to evaluate light attenuation for self-shadowed relighting. To
enhance its training viability and efficiency, we discretize visibility onto a
fixed set of sample directions and supply it with coupled geometric 3D depth
feature and local 2D image feature. We further propose a novel
rendering-inspired loss, namely TransferLoss, to implicitly enforce the
alignment between visibility and occupancy field, enabling end-to-end joint
training. Results and extensive experiments demonstrate the effectiveness of
the proposed method, as it surpasses state-of-the-art in terms of
reconstruction accuracy while achieving comparably accurate relighting to
ray-traced ground truth.

提出了一种新型的稀疏视角下的 3D 重建框架，它通过将占据区域和反照率区域与额外的能见度场密切结合，解决了多视角特征聚合中的遮挡模糊性，能够评估自遮挡重照的光衰减。通过渲染启发式损失，它能够隐式地强制能见度和占据场之间的对齐，使得最终的联合训练效果优越于现有的技术，能够同时实现比光线追踪更为准确的重建和重照。