Handling spatial references in natural language is a key challenge in tasks like autonomous navigation and robotic manipulation. Recent work has investigated various neural architectures for learning multi-modal representations of spatial concepts that generalize well across a variety of observations and text instructions. In this work, we develop accurate models for understanding spatial references in text that are also robust and interpretable. We design a text-conditioned relation network whose parameters are dynamically computed with a cross-modal attention module to capture fine-grained spatial relations between entities. Our experiments across three different prediction tasks demonstrate the effectiveness of our model compared to existing state-of-the-art systems. Our model is robust to both observational and instructional noise, and lends itself to easy interpretation through visualization of intermediate outputs.

本论文提出一种文本条件化的关系网络模型，通过跨模态的注意力机制动态计算参数以捕获实体之间的精细空间关系，从而实现对文本中空间参照的理解，具有可解释性和鲁棒性，在三个任务中实现了17%和15%的表现改进，从而解决了在自主导航和机器人控制中学习空间概念表示的关键挑战。

具有韧性和可解释性的关系网络空间参考基础