Scene graph generation (SGG) aims to parse a visual scene into an
intermediate graph representation for downstream reasoning tasks. Despite
recent advancements, existing methods struggle to generate scene graphs with
novel visual relation concepts. To address this challenge, we introduce a new
open-vocabulary SGG framework based on sequence generation. Our framework
leverages vision-language pre-trained models (VLM) by incorporating an
image-to-graph generation paradigm. Specifically, we generate scene graph
sequences via image-to-text generation with VLM and then construct scene graphs
from these sequences. By doing so, we harness the strong capabilities of VLM
for open-vocabulary SGG and seamlessly integrate explicit relational modeling
for enhancing the VL tasks. Experimental results demonstrate that our design
not only achieves superior performance with an open vocabulary but also
enhances downstream vision-language task performance through explicit relation
modeling knowledge.

基于序列生成的新型开放词汇的场景图生成框架，通过使用视觉语言预训练模型和显式关系建模知识，实现了优质性能的开放词汇感知场景图生成和增强下游视觉语言任务的目的。