This paper introduces CRITICAL, a novel closed-loop framework for autonomous vehicle (AV) training and testing. CRITICAL stands out for its ability to generate diverse scenarios, focusing on critical driving situations that target specific learning and performance gaps identified in the Reinforcement Learning (RL) agent. The framework achieves this by integrating real-world traffic dynamics, driving behavior analysis, surrogate safety measures, and an optional Large Language Model (LLM) component. It is proven that the establishment of a closed feedback loop between the data generation pipeline and the training process can enhance the learning rate during training, elevate overall system performance, and augment safety resilience. Our evaluations, conducted using the Proximal Policy Optimization (PPO) and the HighwayEnv simulation environment, demonstrate noticeable performance improvements with the integration of critical case generation and LLM analysis, indicating CRITICAL's potential to improve the robustness of AV systems and streamline the generation of critical scenarios. This ultimately serves to hasten the development of AV agents, expand the general scope of RL training, and ameliorate validation efforts for AV safety.

本论文介绍了一种名为CRITICAL的新型闭环自主车辆训练和测试框架，通过集成真实世界的交通动态、驾驶行为分析、替代性安全措施和一个可选的大型语言模型（LLM）组件，该框架能够生成各种各样的场景，重点关注强化学习（RL）代理在关键驾驶情况下的学习和性能差距。据证明，在数据生成与训练过程之间建立闭环反馈可以提高训练的学习速率，提升系统整体性能，增强安全弹性。我们的评估使用了近端策略优化（PPO）和HighwayEnv模拟环境进行，证明了关键案例生成和LLM分析的集成可以显著提高性能，表明CRITICAL有潜力提高自主车辆系统的鲁棒性，简化关键场景的生成，从而加快自主车辆代理的开发，扩展强化学习训练的范围，并改善自主车辆安全的验证工作。

通过语言模型整合和关键场景生成增强自主车辆训练