Vision-centric occupancy networks, which represent the surrounding
environment with uniform voxels with semantics, have become a new trend for
safe driving of camera-only autonomous driving perception systems, as they are
able to detect obstacles regardless of their shape and occlusion. Modern
occupancy networks mainly focus on reconstructing visible voxels from object
surfaces with voxel-wise semantic prediction. Usually, they suffer from
inconsistent predictions of one object and mixed predictions for adjacent
objects. These confusions may harm the safety of downstream planning modules.
To this end, we investigate panoptic segmentation on 3D voxel scenarios and
propose an instance-aware occupancy network, PanoSSC. We predict foreground
objects and backgrounds separately and merge both in post-processing. For
foreground instance grouping, we propose a novel 3D instance mask decoder that
can efficiently extract individual objects. we unify geometric reconstruction,
3D semantic segmentation, and 3D instance segmentation into PanoSSC framework
and propose new metrics for evaluating panoptic voxels. Extensive experiments
show that our method achieves competitive results on SemanticKITTI semantic
scene completion benchmark.

通过在三维体素场景中进行全景分割的研究，我们提出了一种实例感知的占据网格模型 PanoSSC，可以有效地提取单个对象，将几何重建、三维语义分割和三维实例分割统一到 PanoSSC 框架中，并提出了用于评估全景体素的新指标。在 SemanticKITTI 语义场景完成基准测试中，我们的方法取得了有竞争力的结果。

PanoSSC：探索用于自动驾驶的单目全景三维场景重构

PanoSSC: Exploring Monocular Panoptic 3D Scene Reconstruction for  Autonomous Driving

When learning common skills like driving, beginners usually have domain
experts standing by to ensure the safety of the learning process. We formulate
such learning scheme under the Expert-in-the-loop Reinforcement Learning where
a guardian is introduced to safeguard the exploration of the learning agent.
While allowing the sufficient exploration in the uncertain environment, the
guardian intervenes under dangerous situations and demonstrates the correct
actions to avoid potential accidents. Thus ERL enables both exploration and
expert's partial demonstration as two training sources. Following such a
setting, we develop a novel Expert Guided Policy Optimization (EGPO) method
which integrates the guardian in the loop of reinforcement learning. The
guardian is composed of an expert policy to generate demonstration and a switch
function to decide when to intervene. Particularly, a constrained optimization
technique is used to tackle the trivial solution that the agent deliberately
behaves dangerously to deceive the expert into taking over. Offline RL
technique is further used to learn from the partial demonstration generated by
the expert. Safe driving experiments show that our method achieves superior
training and test-time safety, outperforms baselines with a substantial margin
in sample efficiency, and preserves the generalizabiliy to unseen environments
in test-time. Demo video and source code are available at:
this https URL

研究者提出了一种基于专家支持的强化学习模式，其中引入了一个守护者来保障学习的过程安全，在保证足够探索性的同时，在危险时进行干预并演示正确行为以避免潜在的事故。使用约束优化技术来避免故意表现出危险行为欺骗专家，并使用离线强化学习技术从专家生成的部分演示中进行学习。试验表明，该方法在训练和测试时表现出更高的安全性，比基线方法在样本效率方面具有更高的性能，并保持对未知环境的一般性。

通过专家指导的策略优化实现安全驾驶

Safe Driving via Expert Guided Policy Optimization

As an initial assessment, over 480,000 labeled virtual images of normal
highway driving were readily generated in Grand Theft Auto V's virtual
environment. Using these images, a CNN was trained to detect following distance
to cars/objects ahead, lane markings, and driving angle (angular heading
relative to lane centerline): all variables necessary for basic autonomous
driving. Encouraging results were obtained when tested on over 50,000 labeled
virtual images from substantially different GTA-V driving environments. This
initial assessment begins to define both the range and scope of the labeled
images needed for training as well as the range and scope of labeled images
needed for testing the definition of boundaries and limitations of trained
networks. It is the efficacy and flexibility of a "GTA-V"-like virtual
environment that is expected to provide an efficient well-defined foundation
for the training and testing of Convolutional Neural Networks for safe driving.
Additionally, described is the Princeton Virtual Environment (PVE) for the
training, testing and enhancement of safe driving AI, which is being developed
using the video-game engine Unity. PVE is being developed to recreate rare but
critical corner cases that can be used in re-training and enhancing machine
learning models and understanding the limitations of current self driving
models. The Florida Tesla crash is being used as an initial reference.

使用 GTA-V 虚拟环境生成 480,000 高速公路驾驶标注图像，训练 CNN 检测基本自动驾驶所需的跟驰距离、车道标线和驾驶角度。使用 Unity-lift 的 PVE 重新创建罕见但关键的转角情况，以重新训练和加强机器学习模型，并理解当前自驾模型的局限性。