Visual Relationship Detection (VRD) has seen significant advancements with
Transformer-based architectures recently. However, we identify two key
limitations in a conventional label assignment for training Transformer-based
VRD models, which is a process of mapping a ground-truth (GT) to a prediction.
Under the conventional assignment, an unspecialized query is trained since a
query is expected to detect every relation, which makes it difficult for a
query to specialize in specific relations. Furthermore, a query is also
insufficiently trained since a GT is assigned only to a single prediction,
therefore near-correct or even correct predictions are suppressed by being
assigned no relation as a GT. To address these issues, we propose Groupwise
Query Specialization and Quality-Aware Multi-Assignment (SpeaQ). Groupwise
Query Specialization trains a specialized query by dividing queries and
relations into disjoint groups and directing a query in a specific query group
solely toward relations in the corresponding relation group. Quality-Aware
Multi-Assignment further facilitates the training by assigning a GT to multiple
predictions that are significantly close to a GT in terms of a subject, an
object, and the relation in between. Experimental results and analyses show
that SpeaQ effectively trains specialized queries, which better utilize the
capacity of a model, resulting in consistent performance gains with zero
additional inference cost across multiple VRD models and benchmarks. Code is
available at this https URL

在此研究中，我们提出使用 Groupwise Query Specialization 和 Quality-Aware Multi-Assignment (SpeaQ) 的方法来解决传统标签分配中的两个问题，从而有效地训练专门化的查询，提高 Visual Relationship Detection (VRD) 的性能。