Reinforcement Learning (RL) plays a crucial role in advancing autonomous
driving technologies by maximizing reward functions to achieve the optimal
policy. However, crafting these reward functions has been a complex, manual
process in many practices. To reduce this complexity, we introduce a novel
framework that integrates Large Language Models (LLMs) with RL to improve
reward function design in autonomous driving. This framework utilizes the
coding capabilities of LLMs, proven in other areas, to generate and evolve
reward functions for highway scenarios. The framework starts with instructing
LLMs to create an initial reward function code based on the driving environment
and task descriptions. This code is then refined through iterative cycles
involving RL training and LLMs' reflection, which benefits from their ability
to review and improve the output. We have also developed a specific prompt
template to improve LLMs' understanding of complex driving simulations,
ensuring the generation of effective and error-free code. Our experiments in a
highway driving simulator across three traffic configurations show that our
method surpasses expert handcrafted reward functions, achieving a 22% higher
average success rate. This not only indicates safer driving but also suggests
significant gains in development productivity.

整合大型语言模型和强化学习的框架提高自动驾驶领域中奖励函数的设计，通过迭代循环的 RL 训练和大型语言模型的反思，生成和优化高速公路场景的奖励函数，实验结果显示该方法相比手工设计的奖励函数能够取得 22% 更高的平均成功率。

使用大型语言模型生成和演化高速行驶的奖励函数

Generating and Evolving Reward Functions for Highway Driving with Large  Language Models

The current autonomous stack is well modularized and consists of perception,
decision making and control in a handcrafted framework. With the advances in
artificial intelligence (AI) and computing resources, researchers have been
pushing the development of end-to-end AI for autonomous driving, at least in
problems of small searching space such as in highway scenarios, and more and
more photorealistic simulation will be critical for efficient learning. In this
research, we propose a novel game-based end-to-end learning and testing
framework for autonomous vehicle highway driving, by learning from human
driving skills. Firstly, we utilize the popular game Grand Theft Auto V (GTA V)
to collect highway driving data with our proposed programmable labels. Then, an
end-to-end architecture predicts the steering and throttle values that control
the vehicle by the image of the game screen. The predicted control values are
sent to the game via a virtual controller to keep the vehicle in lane and avoid
collisions with other vehicles on the road. The proposed solution is validated
in GTA V games, and the results demonstrate the effectiveness of this
end-to-end gamification framework for learning human driving skills.

提出了一种基于游戏的端到端学习和测试框架，通过学习人类驾驶技能，利用 Grand Theft Auto V (GTA V) 游戏中的图像预测车辆的操控值，以实现高速公路驾驶的自主车辆。

端到端训练和测试的游戏化框架用于学习人类高速公路驾驶

End-To-End Training and Testing Gamification Framework to Learn Human  Highway Driving

In highway scenarios, an alert human driver will typically anticipate early
cut-in and cut-out maneuvers of surrounding vehicles using only visual cues. An
automated system must anticipate these situations at an early stage too, to
increase the safety and the efficiency of its performance. To deal with
lane-change recognition and prediction of surrounding vehicles, we pose the
problem as an action recognition/prediction problem by stacking visual cues
from video cameras. Two video action recognition approaches are analyzed:
two-stream convolutional networks and spatiotemporal multiplier networks.
Different sizes of the regions around the vehicles are analyzed, evaluating the
importance of the interaction between vehicles and the context information in
the performance. In addition, different prediction horizons are evaluated. The
obtained results demonstrate the potential of these methodologies to serve as
robust predictors of future lane-changes of surrounding vehicles in time
horizons between 1 and 2 seconds.

利用两种视频动作识别方法对高速公路情境中的自动驾驶变道识别进行了预测，研究结果表明这些方法在 1 到 2 秒的时间范围内预测周围车辆的未来车道变更具有潜在的预测能力。