Visual relation extraction (VRE) aims to extract relations between entities
from visuallyrich documents. Existing methods usually predict relations for
each entity pair independently based on entity features but ignore the global
structure information, i.e., dependencies between entity pairs. The absence of
global structure information may make the model struggle to learn long-range
relations and easily predict conflicted results. To alleviate such limitations,
we propose a GlObal Structure knowledgeguided relation Extraction (GOSE)
framework, which captures dependencies between entity pairs in an iterative
manner. Given a scanned image of the document, GOSE firstly generates
preliminary relation predictions on entity pairs. Secondly, it mines global
structure knowledge based on prediction results of the previous iteration and
further incorporates global structure knowledge into entity representations.
This "generate-capture-incorporate" schema is performed multiple times so that
entity representations and global structure knowledge can mutually reinforce
each other. Extensive experiments show that GOSE not only outperforms previous
methods on the standard fine-tuning setting but also shows promising
superiority in cross-lingual learning; even yields stronger data-efficient
performance in the low-resource setting.

该论文提出了一种 GOSE 框架来进行视觉关系抽取，该框架通过迭代方式捕捉实体对之间的依赖关系以及全局结构知识，并将其融合到实体表示中，实现了实体表示和全局结构知识的相互增强，该方法在标准微调设置下，以及跨语言学习和低资源设置下均表现出有希望的优越性。