Recent work has shown how predictive modeling can endow agents with rich
knowledge of their surroundings, improving their ability to act in complex
environments. We propose question-answering as a general paradigm to decode and
understand the representations that such agents develop, applying our method to
two recent approaches to predictive modeling -action-conditional CPC (Guo et
al., 2018) and SimCore (Gregor et al., 2019). After training agents with these
predictive objectives in a visually-rich, 3D environment with an assortment of
objects, colors, shapes, and spatial configurations, we probe their internal
state representations with synthetic (English) questions, without
backpropagating gradients from the question-answering decoder into the agent.
The performance of different agents when probed this way reveals that they
learn to encode factual, and seemingly compositional, information about
objects, properties and spatial relations from their physical environment. Our
approach is intuitive, i.e. humans can easily interpret responses of the model
as opposed to inspecting continuous vectors, and model-agnostic, i.e.
applicable to any modeling approach. By revealing the implicit knowledge of
objects, quantities, properties and relations acquired by agents as they learn,
question-conditional agent probing can stimulate the design and development of
stronger predictive learning objectives.

通过问答范式破译并理解代理所开发的表示方法，可以揭示出它们从物理环境中学习到的对象、数量、属性和关系的内在知识，并刺激更强大的预测性学习目标的设计和发展。