Learning to collaborate has witnessed significant progress in multi-agent
reinforcement learning (MARL). However, promoting coordination among agents and
enhancing exploration capabilities remain challenges. In multi-agent
environments, interactions between agents are limited in specific situations.
Effective collaboration between agents thus requires a nuanced understanding of
when and how agents' actions influence others. To this end, in this paper, we
propose a novel MARL algorithm named Situation-Dependent Causal Influence-Based
Cooperative Multi-agent Reinforcement Learning (SCIC), which incorporates a
novel Intrinsic reward mechanism based on a new cooperation criterion measured
by situation-dependent causal influence among agents. Our approach aims to
detect inter-agent causal influences in specific situations based on the
criterion using causal intervention and conditional mutual information. This
effectively assists agents in exploring states that can positively impact other
agents, thus promoting cooperation between agents. The resulting update links
coordinated exploration and intrinsic reward distribution, which enhance
overall collaboration and performance. Experimental results on various MARL
benchmarks demonstrate the superiority of our method compared to
state-of-the-art approaches.

提出了一种名为 Situation-Dependent Causal Influence-Based Cooperative Multi-agent Reinforcement Learning (SCIC) 的新型多智能体强化学习算法，通过基于因果关系干预和条件互信息，探测特定情况下智能体间的因果影响，从而促进智能体之间的合作。实验结果表明与其他算法相比，该方法优于目前最先进的方法。

情景依赖的因果影响基础下的合作多智能体强化学习

Situation-Dependent Causal Influence-Based Cooperative Multi-agent  Reinforcement Learning

While autonomous vehicles (AVs) may perform remarkably well in generic
real-life cases, their irrational action in some unforeseen cases leads to
critical safety concerns. This paper introduces the concept of collaborative
reinforcement learning (RL) to generate challenging test cases for AV planning
and decision-making module. One of the critical challenges for collaborative RL
is the credit assignment problem, where a proper assignment of rewards to
multiple agents interacting in the traffic scenario, considering all parameters
and timing, turns out to be non-trivial. In order to address this challenge, we
propose a novel potential-based reward-shaping approach inspired by
counterfactual analysis for solving the credit-assignment problem. The
evaluation in a simulated environment demonstrates the superiority of our
proposed approach against other methods using local and global rewards.

本文介绍了协作强化学习的概念，用于为自主车辆规划和决策模块生成具有挑战性的测试用例，提出了一种基于潜在因素的奖励塑形方法，旨在解决交通场景中多个实体相互作用时的贡献分配问题，并在模拟环境中进行了实验验证。