We build a virtual agent for learning language in a 2D maze-like world. The
agent sees images of the surrounding environment, listens to a virtual teacher,
and takes actions to receive rewards. It interactively learns the teacher's
language from scratch based on two language use cases: sentence-directed
navigation and question answering. It learns simultaneously the visual
representations of the world, the language, and the action control. By
disentangling language grounding from other computational routines and sharing
a concept detection function between language grounding and prediction, the
agent reliably interpolates and extrapolates to interpret sentences that
contain new word combinations or new words missing from training sentences. The
new words are transferred from the answers of language prediction. Such a
language ability is trained and evaluated on a population of over 1.6 million
distinct sentences consisting of 119 object words, 8 color words, 9
spatial-relation words, and 50 grammatical words. The proposed model
significantly outperforms five comparison methods for interpreting zero-shot
sentences. In addition, we demonstrate human-interpretable intermediate outputs
of the model in the appendix.

基于 2D 迷宫世界，通过虚拟代理学习语言的模型，将语言的生成与理解与其他计算流程分离，从而成功地解决新单词出现的问题。模型可以解释人类可理解的中间输出结果，大幅优于其他五种比较方法。