The success of scene graphs for visual scene understanding has brought
attention to the benefits of abstracting a visual input (e.g., image) into a
structured representation, where entities (people and objects) are nodes
connected by edges specifying their relations. Building these representations,
however, requires expensive manual annotation in the form of images paired with
their scene graphs or frames. These formalisms remain limited in the nature of
entities and relations they can capture. In this paper, we propose to leverage
a widely-used meaning representation in the field of natural language
processing, the Abstract Meaning Representation (AMR), to address these
shortcomings. Compared to scene graphs, which largely emphasize spatial
relationships, our visual AMR graphs are more linguistically informed, with a
focus on higher-level semantic concepts extrapolated from visual input.
Moreover, they allow us to generate meta-AMR graphs to unify information
contained in multiple image descriptions under one representation. Through
extensive experimentation and analysis, we demonstrate that we can re-purpose
an existing text-to-AMR parser to parse images into AMRs. Our findings point to
important future research directions for improved scene understanding.

本文提出了使用自然语言处理（NLP）中广泛使用的抽象意义表述（AMR）来建立更具有语言信息的图片抽象表述（称为视觉 AMR 图），并通过重复试验和分析表明，我们可以重用现有的文本到 AMR 分析器来分析图像到 AMRs 的转换。