Most of the prior work on multi-agent reinforcement learning (MARL) achieves
optimal collaboration by directly controlling the agents to maximize a common
reward. In this paper, we aim to address this from a different angle. In
particular, we consider scenarios where there are self-interested agents (i.e.,
worker agents) which have their own minds (preferences, intentions, skills,
etc.) and can not be dictated to perform tasks they do not wish to do. For
achieving optimal coordination among these agents, we train a super agent
(i.e., the manager) to manage them by first inferring their minds based on both
current and past observations and then initiating contracts to assign suitable
tasks to workers and promise to reward them with corresponding bonuses so that
they will agree to work together. The objective of the manager is maximizing
the overall productivity as well as minimizing payments made to the workers for
ad-hoc worker teaming. To train the manager, we propose Mind-aware Multi-agent
Management Reinforcement Learning (M^3RL), which consists of agent modeling and
policy learning. We have evaluated our approach in two environments, Resource
Collection and Crafting, to simulate multi-agent management problems with
various task settings and multiple designs for the worker agents. The
experimental results have validated the effectiveness of our approach in
modeling worker agents' minds online, and in achieving optimal ad-hoc teaming
with good generalization and fast adaptation.

本文提出 Mind-aware Multi-agent Management Reinforcement Learning (M^3RL) 算法，通过对多个自我意识且不可控制的机器人进行建模，实现超级机器人的管理和操纵，以实现最优的多机器人协同任务分配和奖励系统。