Generative AI is increasingly important in software engineering, including
safety engineering, where its use ensures that software does not cause harm to
people. This also leads to high quality requirements for generative AI.
Therefore, the simplistic use of Large Language Models (LLMs) alone will not
meet these quality demands. It is crucial to develop more advanced and
sophisticated approaches that can effectively address the complexities and
safety concerns of software systems. Ultimately, humans must understand and
take responsibility for the suggestions provided by generative AI to ensure
system safety. To this end, we present an efficient, hybrid strategy to
leverage LLMs for safety analysis and Human-AI codesign. In particular, we
develop a customized LLM agent that uses elements of prompt engineering,
heuristic reasoning, and retrieval-augmented generation to solve tasks
associated with predefined safety concepts, in interaction with a system model
graph. The reasoning is guided by a cascade of micro-decisions that help
preserve structured information. We further suggest a graph verbalization which
acts as an intermediate representation of the system model to facilitate
LLM-graph interactions. Selected pairs of prompts and responses relevant for
safety analytics illustrate our method for the use case of a simplified
automated driving system.

通过定制化的大型语言模型 (LLM) 代理，采用提示工程、启发式推理和检索增强生成的元素，以保持结构化信息，我们提出了一种高效的混合策略，利用 LLM 进行安全分析和人工智能与人类的协同设计，以提高软件工程和安全工程中的生成式 AI 的质量要求，并确保人类对生成式 AI 提供的建议负责，通过图形表述作为系统模型的中间表示，促进了 LLM 和图形之间的交互，以简化的自动驾驶系统为案例说明了我们的方法。