In human-AI interaction, a prominent goal is to attain human`s desirable
outcome with the assistance of AI agents, which can be ideally delineated as a
problem of seeking the optimal Nash Equilibrium that matches the human`s
desirable outcome. However, reaching the outcome is usually challenging due to
the existence of multiple Nash Equilibria that are related to the assisting
task but do not correspond to the human`s desirable outcome. To tackle this
issue, we employ a theoretical framework called structural causal game (SCG) to
formalize the human-AI interactive process. Furthermore, we introduce a
strategy referred to as pre-policy intervention on the SCG to steer AI agents
towards attaining the human`s desirable outcome. In more detail, a pre-policy
is learned as a generalized intervention to guide the agents` policy selection,
under a transparent and interpretable procedure determined by the SCG. To make
the framework practical, we propose a reinforcement learning-like algorithm to
search out this pre-policy. The proposed algorithm is tested in both gridworld
environments and realistic dialogue scenarios with large language models,
demonstrating its adaptability in a broader class of problems and potential
effectiveness in real-world situations.

人工智能与人类互动中，通过结构因果博弈的理论框架提出了一种预策略干预算法，以引导人工智能代理实现人类的期望结果，并通过强化学习算法在格子世界环境和实际对话场景中进行了测试，展示出其在更广泛问题和真实世界情景中的适应能力和潜在有效性。