Today's scene graph generation (SGG) task is still far from practical, mainly
due to the severe training bias, e.g., collapsing diverse "human walk on / sit
on / lay on beach" into "human on beach". Given such SGG, the down-stream tasks
such as VQA can hardly infer better scene structures than merely a bag of
objects. However, debiasing in SGG is not trivial because traditional debiasing
methods cannot distinguish between the good and bad bias, e.g., good context
prior (e.g., "person read book" rather than "eat") and bad long-tailed bias
(e.g., "near" dominating "behind / in front of"). In this paper, we present a
novel SGG framework based on causal inference but not the conventional
likelihood. We first build a causal graph for SGG, and perform traditional
biased training with the graph. Then, we propose to draw the counterfactual
causality from the trained graph to infer the effect from the bad bias, which
should be removed. In particular, we use Total Direct Effect (TDE) as the
proposed final predicate score for unbiased SGG. Note that our framework is
agnostic to any SGG model and thus can be widely applied in the community who
seeks unbiased predictions. By using the proposed Scene Graph Diagnosis toolkit
on the SGG benchmark Visual Genome and several prevailing models, we observed
significant improvements over the previous state-of-the-art methods.

本文通过因果推断方法构建了一个场景图生成的因果图，并通过传统偏差训练和反事实因果关系推断方法去除其不良偏差，最终提出了一个基于 Total Direct Effect 的无偏差 SGG 框架。