Although experience sharing (ES) accelerates multiagent reinforcement learning (MARL) in an advisor-advisee framework, attempts to apply ES to decentralized multiagent systems have so far relied on trusted environments and overlooked the possibility of adversarial manipulation and inference. Nevertheless, in a real-world setting, some Byzantine attackers, disguised as advisors, may provide false advice to the advisee and catastrophically degrade the overall learning performance. Also, an inference attacker, disguised as an advisee, may conduct several queries to infer the advisors' private information and make the entire ES process questionable in terms of privacy leakage. To address and tackle these issues, we propose a novel MARL framework (BRNES) that heuristically selects a dynamic neighbor zone for each advisee at each learning step and adopts a weighted experience aggregation technique to reduce Byzantine attack impact. Furthermore, to keep the agent's private information safe from adversarial inference attacks, we leverage the local differential privacy (LDP)-induced noise during the ES process. Our experiments show that our framework outperforms the state-of-the-art in terms of the steps to goal, obtained reward, and time to goal metrics. Particularly, our evaluation shows that the proposed framework is 8.32x faster than the current non-private frameworks and 1.41x faster than the private frameworks in an adversarial setting.

我们提出了一种新颖的多智能体强化学习（MARL）框架（BRNES），通过启发式地为每个受劝者选择动态邻居区域，采用加权的经验聚合技术来减少拜占庭攻击的影响，此外，在 ES 过程中通过本地差分隐私（LDP）导致的噪声保护代理的私人信息，我们的实验证明，我们的框架在目标步骤数、获得奖励和达到目标的时间指标上优于现有技术。特别是，在敌对环境中，我们的评估结果显示，该提出的框架比当前的非私密框架快 8.32 倍，比私密框架快 1.41 倍。

BRNES：多体机器人和自主系统中的安全与隐私感知式体验共享