Covering option discovery has been developed to improve the exploration of
reinforcement learning in single-agent scenarios with sparse reward signals,
through connecting the most distant states in the embedding space provided by
the Fiedler vector of the state transition graph. However, these option
discovery methods cannot be directly extended to multi-agent scenarios, since
the joint state space grows exponentially with the number of agents in the
system. Thus, existing researches on adopting options in multi-agent scenarios
still rely on single-agent option discovery and fail to directly discover the
joint options that can improve the connectivity of the joint state space of
agents. In this paper, we show that it is indeed possible to directly compute
multi-agent options with collaborative exploratory behaviors among the agents,
while still enjoying the ease of decomposition. Our key idea is to approximate
the joint state space as a Kronecker graph -- the Kronecker product of
individual agents' state transition graphs, based on which we can directly
estimate the Fiedler vector of the joint state space using the Laplacian
spectrum of individual agents' transition graphs. This decomposition enables us
to efficiently construct multi-agent joint options by encouraging agents to
connect the sub-goal joint states which are corresponding to the minimum or
maximum values of the estimated joint Fiedler vector. The evaluation based on
multi-agent collaborative tasks shows that the proposed algorithm can
successfully identify multi-agent options, and significantly outperforms prior
works using single-agent options or no options, in terms of both faster
exploration and higher cumulative rewards.

本文提出了一种基于 Kronecker 图的多智能体协同探索的选项发现方法，通过鼓励智能体连接相应的最小或最大 Fiedler 向量，构建多智能体共同目标状态，从而在多智能体任务中实现更快的探索和更高的累积奖励。

使用因子图为基础的表格强化学习中学习多智能体选项

Learning Multi-agent Options for Tabular Reinforcement Learning using Factor Graphs

ScoutBot is a dialogue interface to physical and simulated robots that
supports collaborative exploration of environments. The demonstration will
allow users to issue unconstrained spoken language commands to ScoutBot.
ScoutBot will prompt for clarification if the user's instruction needs
additional input. It is trained on human-robot dialogue collected from
Wizard-of-Oz experiments, where robot responses were initiated by a human
wizard in previous interactions. The demonstration will show a simulated ground
robot (Clearpath Jackal) in a simulated environment supported by ROS (Robot
Operating System).

ScoutBot 是一个对话界面，可支持机器人的协同探测环境，可以通过 ROS 对 Clearpath Jackal 进行仿真，并通过 Wizard-of-Oz 实验人机对话数据训练来支持基于口语的命令输入。