We analyze the language learned by an agent trained with reinforcement
learning as a component of the ActiveQA system [Buck et al., 2017]. In
ActiveQA, question answering is framed as a reinforcement learning task in
which an agent sits between the user and a black box question-answering system.
The agent learns to reformulate the user's questions to elicit the optimal
answers. It probes the system with many versions of a question that are
generated via a sequence-to-sequence question reformulation model, then
aggregates the returned evidence to find the best answer. This process is an
instance of \emph{machine-machine} communication. The question reformulation
model must adapt its language to increase the quality of the answers returned,
matching the language of the question answering system. We find that the agent
does not learn transformations that align with semantic intuitions but
discovers through learning classical information retrieval techniques such as
tf-idf re-weighting and stemming.

该文章分析了一个被训练使用强化学习的代理程序学到的语言，作为 ActiveQA 系统的组成部分。在这个系统中，问题回答被框架化为一项强化学习任务，代理学习重构用户问题以引出最佳答案。研究发现，代理程序并没有学习与语义相关的转换，而是通过学习传统的信息检索技术（如 tf-idf 重新加权和词干化）来适应回答系统的语言。