Dynamic scene graph generation (SGG) focuses on detecting objects in a video
and determining their pairwise relationships. Existing dynamic SGG methods
usually suffer from several issues, including 1) Contextual noise, as some
frames might contain occluded and blurred objects. 2) Label bias, primarily due
to the high imbalance between a few positive relationship samples and numerous
negative ones. Additionally, the distribution of relationships exhibits a
long-tailed pattern. To address the above problems, in this paper, we introduce
a network named TD$^2$-Net that aims at denoising and debiasing for dynamic
SGG. Specifically, we first propose a denoising spatio-temporal transformer
module that enhances object representation with robust contextual information.
This is achieved by designing a differentiable Top-K object selector that
utilizes the gumbel-softmax sampling strategy to select the relevant
neighborhood for each object. Second, we introduce an asymmetrical reweighting
loss to relieve the issue of label bias. This loss function integrates
asymmetry focusing factors and the volume of samples to adjust the weights
assigned to individual samples. Systematic experimental results demonstrate the
superiority of our proposed TD$^2$-Net over existing state-of-the-art
approaches on Action Genome databases. In more detail, TD$^2$-Net outperforms
the second-best competitors by 12.7 \% on mean-Recall@10 for predicate
classification.

本文介绍了一种名为 TD$^2$-Net 的网络，旨在进行动态场景图生成时的去噪和去偏执。该网络通过设计可微分的 Top-K 对象选择器来增强对象表示，以及引入不对称重新加权损失函数来缓解标签偏差问题。在行动基因组数据库上的系统性实验结果表明，我们提出的 TD$^2$-Net 相对于现有最先进的方法在谓词分类的均值召回率 @10 上超过最优竞争对手 12.7%。