Before taking actions in an environment with more than one intelligent agent,
an autonomous agent may benefit from reasoning about the other agents and
utilizing a notion of a guarantee or confidence about the behavior of the
system. In this article, we propose a novel multi-agent reinforcement learning
(MARL) algorithm CAMMARL, which involves modeling the actions of other agents
in different situations in the form of confident sets, i.e., sets containing
their true actions with a high probability. We then use these estimates to
inform an agent's decision-making. For estimating such sets, we use the concept
of conformal predictions, by means of which, we not only obtain an estimate of
the most probable outcome but get to quantify the operable uncertainty as well.
For instance, we can predict a set that provably covers the true predictions
with high probabilities (e.g., 95%). Through several experiments in two fully
cooperative multi-agent tasks, we show that CAMMARL elevates the capabilities
of an autonomous agent in MARL by modeling conformal prediction sets over the
behavior of other agents in the environment and utilizing such estimates to
enhance its policy learning. All developed codes can be found here:
this https URL

本文提出了一种基于置信集和符合预测的多智能体强化学习算法 CAMMARL，并通过在两个完全协作的多智能体任务中进行的多个实验证明了该算法可以提高智能体的策略学习能力。