Photorealistic simulation plays a crucial role in applications such as
autonomous driving, where advances in neural radiance fields (NeRFs) may allow
better scalability through the automatic creation of digital 3D assets.
However, reconstruction quality suffers on street scenes due to largely
collinear camera motions and sparser samplings at higher speeds. On the other
hand, the application often demands rendering from camera views that deviate
from the inputs to accurately simulate behaviors like lane changes. In this
paper, we propose several insights that allow a better utilization of Lidar
data to improve NeRF quality on street scenes. First, our framework learns a
geometric scene representation from Lidar, which is fused with the implicit
grid-based representation for radiance decoding, thereby supplying stronger
geometric information offered by explicit point cloud. Second, we put forth a
robust occlusion-aware depth supervision scheme, which allows utilizing
densified Lidar points by accumulation. Third, we generate augmented training
views from Lidar points for further improvement. Our insights translate to
largely improved novel view synthesis under real driving scenes.

利用 Lidar 数据改善街景下基于神经辐射场的光线追踪和视图合成技术。

DiL-NeRF：透视街景中的激光雷达神经辐射场探究

DiL-NeRF: Delving into Lidar for Neural Radiance Field on Street Scenes

The current autonomous stack is well modularized and consists of perception,
decision making and control in a handcrafted framework. With the advances in
artificial intelligence (AI) and computing resources, researchers have been
pushing the development of end-to-end AI for autonomous driving, at least in
problems of small searching space such as in highway scenarios, and more and
more photorealistic simulation will be critical for efficient learning. In this
research, we propose a novel game-based end-to-end learning and testing
framework for autonomous vehicle highway driving, by learning from human
driving skills. Firstly, we utilize the popular game Grand Theft Auto V (GTA V)
to collect highway driving data with our proposed programmable labels. Then, an
end-to-end architecture predicts the steering and throttle values that control
the vehicle by the image of the game screen. The predicted control values are
sent to the game via a virtual controller to keep the vehicle in lane and avoid
collisions with other vehicles on the road. The proposed solution is validated
in GTA V games, and the results demonstrate the effectiveness of this
end-to-end gamification framework for learning human driving skills.

提出了一种基于游戏的端到端学习和测试框架，通过学习人类驾驶技能，利用 Grand Theft Auto V (GTA V) 游戏中的图像预测车辆的操控值，以实现高速公路驾驶的自主车辆。

端到端训练和测试的游戏化框架用于学习人类高速公路驾驶

End-To-End Training and Testing Gamification Framework to Learn Human  Highway Driving

Drones equipped with cameras are emerging as a powerful tool for large-scale
aerial 3D scanning, but existing automatic flight planners do not exploit all
available information about the scene, and can therefore produce inaccurate and
incomplete 3D models. We present an automatic method to generate drone
trajectories, such that the imagery acquired during the flight will later
produce a high-fidelity 3D model. Our method uses a coarse estimate of the
scene geometry to plan camera trajectories that: (1) cover the scene as
thoroughly as possible; (2) encourage observations of scene geometry from a
diverse set of viewing angles; (3) avoid obstacles; and (4) respect a
user-specified flight time budget. Our method relies on a mathematical model of
scene coverage that exhibits an intuitive diminishing returns property known as
submodularity. We leverage this property extensively to design a trajectory
planning algorithm that reasons globally about the non-additive coverage reward
obtained across a trajectory, jointly with the cost of traveling between views.
We evaluate our method by using it to scan three large outdoor scenes, and we
perform a quantitative evaluation using a photorealistic video game simulator.

该论文介绍了一种利用数学模型自动生成无人机飞行轨迹，全面准确地扫描物体表面，从而生成高保真的 3D 模型。作者使用了覆盖度等数学原理，进行路径规划，可以自动权衡机器人特性和覆盖度，有效避免撞墙等情况。