Transformer-based detection and segmentation methods use a list of learned
detection queries to retrieve information from the transformer network and
learn to predict the location and category of one specific object from each
query. We empirically find that random convex combinations of the learned
queries are still good for the corresponding models. We then propose to learn a
convex combination with dynamic coefficients based on the high-level semantics
of the image. The generated dynamic queries, named modulated queries, better
capture the prior of object locations and categories in the different images.
Equipped with our modulated queries, a wide range of DETR-based models achieve
consistent and superior performance across multiple tasks including object
detection, instance segmentation, panoptic segmentation, and video instance
segmentation.

使用基于 Transformer 的检测和分割方法，学习检测查询列表以从变压器网络中检索信息，并学习预测每个查询中一个特定对象的位置和类别。通过学习基于图像的高层语义的动态系数的凸组合的方式，生成的动态查询更好地捕获不同图像中对象位置和类别的先验信息。利用我们的模态查询，一系列基于 DETR 的模型在多个任务中实现了一致且优秀的性能，包括物体检测，实例分割，全景分割和视频实例分割。

DQ-Det：基于 Transformer 的物体检测与分割的学习动态查询组合

DQ-Det: Learning Dynamic Query Combinations for Transformer-based Object  Detection and Segmentation

Object localization in general environments is a fundamental part of vision
systems. While dominating on the COCO benchmark, recent Transformer-based
detection methods are not competitive in diverse domains. Moreover, these
methods still struggle to very accurately estimate the object bounding boxes in
complex environments.
We introduce Cascade-DETR for high-quality universal object detection. We
jointly tackle the generalization to diverse domains and localization accuracy
by proposing the Cascade Attention layer, which explicitly integrates
object-centric information into the detection decoder by limiting the attention
to the previous box prediction. To further enhance accuracy, we also revisit
the scoring of queries. Instead of relying on classification scores, we predict
the expected IoU of the query, leading to substantially more well-calibrated
confidences. Lastly, we introduce a universal object detection benchmark,
UDB10, that contains 10 datasets from diverse domains. While also advancing the
state-of-the-art on COCO, Cascade-DETR substantially improves DETR-based
detectors on all datasets in UDB10, even by over 10 mAP in some cases. The
improvements under stringent quality requirements are even more pronounced. Our
code and models will be released at this https URL

我们引入了 Cascade-DETR 用于高质量的通用目标检测，通过提出级联注意力层来共同解决对多样领域的泛化和定位准确性问题，通过限制关注先前的目标框预测来显式地将对象中心信息集成到检测解码器中。为了进一步提高准确性，我们重新审视了查询的评分，不依赖于分类分数，而是预测查询的预期 IoU，从而显著提高了置信度的校准性，最后，我们引入了一个包含来自多个领域的 10 个数据集的通用目标检测基准，UDB10，在 COCO 上也取得了最新的进展，Cascade-DETR 在 UDB10 的所有数据集上都对基于 DETR 的检测器有了显著的改进，有些甚至超过 10 个 mAP，严格质量要求下的改进效果更加显著。