Being able to understand visual scenes is a precursor for many downstream
tasks, including autonomous driving, robotics, and other vision-based
approaches. A common approach enabling the ability to reason over visual data
is Scene Graph Generation (SGG); however, many existing approaches assume
undisturbed vision, i.e., the absence of real-world corruptions such as fog,
snow, smoke, as well as non-uniform perturbations like sun glare or water
drops. In this work, we propose a novel SGG benchmark containing procedurally
generated weather corruptions and other transformations over the Visual Genome
dataset. Further, we introduce a corresponding approach, Hierarchical Knowledge
Enhanced Robust Scene Graph Generation (HiKER-SGG), providing a strong baseline
for scene graph generation under such challenging setting. At its core,
HiKER-SGG utilizes a hierarchical knowledge graph in order to refine its
predictions from coarse initial estimates to detailed predictions. In our
extensive experiments, we show that HiKER-SGG does not only demonstrate
superior performance on corrupted images in a zero-shot manner, but also
outperforms current state-of-the-art methods on uncorrupted SGG tasks. Code is
available at this https URL

本研究提出了一种新的 SGG 基准测试方法，包括过程生成的天气损坏和其它转换，提出了一种称为 HiKER-SGG 的方法，利用层次化知识图对场景图生成进行精化预测，并在苛刻环境下展示了卓越的性能。