Although pre-trained language models encode generic knowledge beneficial for
planning and control, they may fail to generate appropriate control policies
for domain-specific tasks. Existing fine-tuning methods use human feedback to
address this limitation, however, sourcing human feedback is labor intensive
and costly. We present a fully automated approach to fine-tune pre-trained
language models for applications in autonomous systems, bridging the gap
between generic knowledge and domain-specific requirements while reducing cost.
The method synthesizes automaton-based controllers from pre-trained models
guided by natural language task descriptions. These controllers are verifiable
against independently provided specifications within a world model, which can
be abstract or obtained from a high-fidelity simulator. Controllers with high
compliance with the desired specifications receive higher ranks, guiding the
iterative fine-tuning process. We provide quantitative evidences, primarily in
autonomous driving, to demonstrate the method's effectiveness across multiple
tasks. The results indicate an improvement in percentage of specifications
satisfied by the controller from 60% to 90%.

使用自然语言任务描述，通过自动合成基于自动机的控制器，并通过与独立提供的规范进行验证，将预训练语言模型完全自动调优以适应自主系统的应用，从而在减少成本的同时弥补了通用知识和特定领域要求之间的差距，其在自主驾驶等多个任务中显示出有效性，使控制器所满足规范的百分比从 60% 提高到 90%。