Service robots should be able to interact naturally with non-expert human
users, not only to help them in various tasks but also to receive guidance in
order to resolve ambiguities that might be present in the instruction. We
consider the task of visual grounding, where the agent segments an object from
a crowded scene given a natural language description. Modern holistic
approaches to visual grounding usually ignore language structure and struggle
to cover generic domains, therefore relying heavily on large datasets.
Additionally, their transfer performance in RGB-D datasets suffers due to high
visual discrepancy between the benchmark and the target domains. Modular
approaches marry learning with domain modeling and exploit the compositional
nature of language to decouple visual representation from language parsing, but
either rely on external parsers or are trained in an end-to-end fashion due to
the lack of strong supervision. In this work, we seek to tackle these
limitations by introducing a fully decoupled modular framework for
compositional visual grounding of entities, attributes, and spatial relations.
We exploit rich scene graph annotations generated in a synthetic domain and
train each module independently. Our approach is evaluated both in simulation
and in two real RGB-D scene datasets. Experimental results show that the
decoupled nature of our framework allows for easy integration with domain
adaptation approaches for Sim-To-Real visual recognition, offering a
data-efficient, robust, and interpretable solution to visual grounding in
robotic applications.

本研究介绍了一种基于模块化方法的场景感知机器人视觉定位框架，该方法通过训练每个模块来独立解析实体、属性和空间关系，并结合领域自适应技术解决常见问题。实验表明，该框架与 Sim-To-Real 实现的视觉识别方法相结合，能够提供一种数据高效、稳健且易于理解的视觉定位机器人方案。