To generate "accurate" scene graphs, almost all existing methods predict
pairwise relationships in a deterministic manner. However, we argue that visual
relationships are often semantically ambiguous. Specifically, inspired by
linguistic knowledge, we classify the ambiguity into three types: Synonymy
Ambiguity, Hyponymy Ambiguity, and Multi-view Ambiguity. The ambiguity
naturally leads to the issue of \emph{implicit multi-label}, motivating the
need for diverse predictions. In this work, we propose a novel plug-and-play
Probabilistic Uncertainty Modeling (PUM) module. It models each union region as
a Gaussian distribution, whose variance measures the uncertainty of the
corresponding visual content. Compared to the conventional deterministic
methods, such uncertainty modeling brings stochasticity of feature
representation, which naturally enables diverse predictions. As a byproduct,
PUM also manages to cover more fine-grained relationships and thus alleviates
the issue of bias towards frequent relationships. Extensive experiments on the
large-scale Visual Genome benchmark show that combining PUM with newly proposed
ResCAGCN can achieve state-of-the-art performances, especially under the mean
recall metric. Furthermore, we prove the universal effectiveness of PUM by
plugging it into some existing models and provide insightful analysis of its
ability to generate diverse yet plausible visual relationships.

本研究提出了一种新的 Probabilistic Uncertainty Modeling（PUM）模块，通过对不确定性建模实现具有多样性的场景图预测，并与 ResCAGCN 模型相结合，在 Visual Genome 基准测试中取得了最先进的性能。