Scene Graph Generation (SGG) can extract abstract semantic relations between
entities in images as graph representations. This task holds strong promises
for other downstream tasks such as the embodied cognition of an autonomous
agent. However, to power such applications, SGG needs to solve the gap of
real-time latency. In this work, we propose to investigate the bottlenecks of
current approaches for real-time constraint applications. Then, we propose a
simple yet effective implementation of a real-time SGG approach using YOLOV8 as
an object detection backbone. Our implementation is the first to obtain more
than 48 FPS for the task with no loss of accuracy, successfully outperforming
any other lightweight approaches. Our code is freely available at
this https URL

我们提出了一种使用 YOLOV8 作为目标检测骨干网络的实时场景图生成方法，该方法在任务中首次实现超过 48 FPS 的速度，且没有准确度损失，成功胜过任何其他轻量级方法。

实时场景图生成

Real-Time Scene Graph Generation

Unbiased SGG has achieved significant progress over recent years. However,
almost all existing SGG models have overlooked the ground-truth annotation
qualities of prevailing SGG datasets, i.e., they always assume: 1) all the
manually annotated positive samples are equally correct; 2) all the
un-annotated negative samples are absolutely background. In this paper, we
argue that both assumptions are inapplicable to SGG: there are numerous "noisy"
groundtruth predicate labels that break these two assumptions, and these noisy
samples actually harm the training of unbiased SGG models. To this end, we
propose a novel model-agnostic NoIsy label CorrEction strategy for SGG: NICE.
NICE can not only detect noisy samples but also reassign more high-quality
predicate labels to them. After the NICE training, we can obtain a cleaner
version of SGG dataset for model training. Specifically, NICE consists of three
components: negative Noisy Sample Detection (Neg-NSD), positive NSD (Pos-NSD),
and Noisy Sample Correction (NSC). Firstly, in Neg-NSD, we formulate this task
as an out-of-distribution detection problem, and assign pseudo labels to all
detected noisy negative samples. Then, in Pos-NSD, we use a clustering-based
algorithm to divide all positive samples into multiple sets, and treat the
samples in the noisiest set as noisy positive samples. Lastly, in NSC, we use a
simple but effective weighted KNN to reassign new predicate labels to noisy
positive samples. Extensive results on different backbones and tasks have
attested to the effectiveness and generalization abilities of each component of
NICE.

该论文提出了一种用于物体间关系检测的名为 NICE 的模型，并通过上述三个组件解决了存在于 SGG 数据集中标签质量低下的问题。