We approach the problem of understanding how people interact with each other
in collaborative settings, especially when individuals know little about their
teammates, via Multiagent Inverse Reinforcement Learning (MIRL), where the goal
is to infer the reward functions guiding the behavior of each individual given
trajectories of a team's behavior during some task. Unlike current MIRL
approaches, we do not assume that team members know each other's goals a
priori; rather, that they collaborate by adapting to the goals of others
perceived by observing their behavior, all while jointly performing a task. To
address this problem, we propose a novel approach to MIRL via Theory of Mind
(MIRL-ToM). For each agent, we first use ToM reasoning to estimate a posterior
distribution over baseline reward profiles given their demonstrated behavior.
We then perform MIRL via decentralized equilibrium by employing single-agent
Maximum Entropy IRL to infer a reward function for each agent, where we
simulate the behavior of other teammates according to the time-varying
distribution over profiles. We evaluate our approach in a simulated 2-player
search-and-rescue operation where the goal of the agents, playing different
roles, is to search for and evacuate victims in the environment. Our results
show that the choice of baseline profiles is paramount to the recovery of the
ground-truth rewards, and that MIRL-ToM is able to recover the rewards used by
agents interacting both with known and unknown teammates.

本文探讨了在协作场景中人们如何相互交互尤其是在个体了解队友很少的情况下，通过多智能体逆强化学习（MIRL）来推断每个个体行为背后的奖励函数。针对这个问题，我们提出一个新颖的 MIRL-ToM 模型，结合了理论思维（Theory of Mind）和最大熵 IRL，成功地恢复了用于有知晓或无知晓队友互动的奖励。

基于心智推理的多智能体逆强化学习

Multiagent Inverse Reinforcement Learning via Theory of Mind Reasoning

We address the problem of localizing non-collaborative WiFi devices in a
large region. Our main motive is to localize humans by localizing their WiFi
devices, e.g. during search-and-rescue operations after a natural disaster. We
use an active sensing approach that relies on Unmanned Aerial Vehicles (UAVs)
to collect signal-strength measurements at informative locations. The problem
is challenging since the measurement is received at arbitrary times and they
are received only when the UAV is in close proximity to the device. For these
reasons, it is extremely important to make prudent decision with very few
measurements. We use the Bayesian optimization approach based on Gaussian
process (GP) regression. This approach works well for our application since GPs
give reliable predictions with very few measurements while Bayesian
optimization makes a judicious trade-off between exploration and exploitation.
In field experiments conducted over a region of 1000 $\times$ 1000 $m^2$, we
show that our approach reduces the search area to less than 100 meters around
the WiFi device within 5 minutes only. Overall, our approach localizes the
device in less than 15 minutes with an error of less than 20 meters.

使用基于贝叶斯优化的高斯过程回归方法，结合无人机采集的信号强度测量，可以在灾后搜救等应用场景下，仅仅花费少量时间和测量准确地定位 WiFi 设备的位置。