We frame Question Answering (QA) as a Reinforcement Learning task, an
approach that we call Active Question Answering. We propose an agent that sits
between the user and a black box QA system and learns to reformulate questions
to elicit the best possible answers. The agent probes the system with,
potentially many, natural language reformulations of an initial question and
aggregates the returned evidence to yield the best answer. The reformulation
system is trained end-to-end to maximize answer quality using policy gradient.
We evaluate on SearchQA, a dataset of complex questions extracted from
Jeopardy!. The agent outperforms a state-of-the-art base model, playing the
role of the environment, and other benchmarks. We also analyze the language
that the agent has learned while interacting with the question answering
system. We find that successful question reformulations look quite different
from natural language paraphrases. The agent is able to discover non-trivial
reformulation strategies that resemble classic information retrieval techniques
such as term re-weighting (tf-idf) and stemming.

本文介绍了一种称为 Active Question Answering 的基于强化学习的问题回答方法，通过探究黑盒 QA 系统并最大化回答质量来重新构筑问题，该方法在 Jeopardy! 等复杂问题数据集上表现出比其他基准模型更好的性能，并发现了与信息检索技术类似的策略。