Scene graph generation aims to capture detailed spatial and semantic
relationships between objects in an image, which is challenging due to
incomplete labelling, long-tailed relationship categories, and relational
semantic overlap. Existing Transformer-based methods either employ distinct
queries for objects and predicates or utilize holistic queries for relation
triplets and hence often suffer from limited capacity in learning low-frequency
relationships. In this paper, we present a new Transformer-based method, called
DSGG, that views scene graph detection as a direct graph prediction problem
based on a unique set of graph-aware queries. In particular, each graph-aware
query encodes a compact representation of both the node and all of its
relations in the graph, acquired through the utilization of a relaxed sub-graph
matching during the training process. Moreover, to address the problem of
relational semantic overlap, we utilize a strategy for relation distillation,
aiming to efficiently learn multiple instances of semantic relationships.
Extensive experiments on the VG and the PSG datasets show that our model
achieves state-of-the-art results, showing a significant improvement of 3.5\%
and 6.7\% in mR@50 and mR@100 for the scene-graph generation task and achieves
an even more substantial improvement of 8.5\% and 10.3\% in mR@50 and mR@100
for the panoptic scene graph generation task. Code is available at
https://github.com/zeeshanhayder/DSGG.

本文介绍了一种新的基于 Transformer 的方法，称为 DSGG，将场景图检测视为基于一组独特的图感知查询的直接图预测问题，并采用放松子图匹配的方式获取图节点及其关系的紧凑表示，以及通过关系蒸馏策略来解决关系语义重叠问题。实验结果表明，该模型在场景图生成任务上取得了最先进的结果，在 mR@50 和 mR@100 方面分别取得了 3.5％和 6.7％的显著改进，在全景场景图生成任务上甚至取得了 8.5％和 10.3％的更大改进。

DSGG：稠密关系变换器用于端到端场景图生成

DSGG: Dense Relation Transformer for an End-to-end Scene Graph  Generation

Available 3D human pose estimation approaches leverage different forms of
strong (2D/3D pose) or weak (multi-view or depth) paired supervision. Barring
synthetic or in-studio domains, acquiring such supervision for each new target
environment is highly inconvenient. To this end, we cast 3D pose learning as a
self-supervised adaptation problem that aims to transfer the task knowledge
from a labeled source domain to a completely unpaired target. We propose to
infer image-to-pose via two explicit mappings viz. image-to-latent and
latent-to-pose where the latter is a pre-learned decoder obtained from a
prior-enforcing generative adversarial auto-encoder. Next, we introduce
relation distillation as a means to align the unpaired cross-modal samples i.e.
the unpaired target videos and unpaired 3D pose sequences. To this end, we
propose a new set of non-local relations in order to characterize long-range
latent pose interactions unlike general contrastive relations where positive
couplings are limited to a local neighborhood structure. Further, we provide an
objective way to quantify non-localness in order to select the most effective
relation set. We evaluate different self-adaptation settings and demonstrate
state-of-the-art 3D human pose estimation performance on standard benchmarks.

本研究提出了一种自我监督的适应方法，将 3D 姿势学习转换为源域标记到未标记目标域的知识转移，通过两个显式映射推断图像到姿势，提出关系蒸馏作为一种方法来对齐未配对的跨模态样本，并使用新的非局部关系来表征长程潜在姿态相互作用，在标准基准测试中展示了最先进的 3D 人体姿态估计性能。