Transformer-based detection and segmentation methods use a list of learned detection queries to retrieve information from the transformer network and learn to predict the location and category of one specific object from each query. We empirically find that random convex combinations of the learned queries are still good for the corresponding models. We then propose to learn a convex combination with dynamic coefficients based on the high-level semantics of the image. The generated dynamic queries, named modulated queries, better capture the prior of object locations and categories in the different images. Equipped with our modulated queries, a wide range of DETR-based models achieve consistent and superior performance across multiple tasks including object detection, instance segmentation, panoptic segmentation, and video instance segmentation.

使用基于Transformer的检测和分割方法，学习检测查询列表以从变压器网络中检索信息，并学习预测每个查询中一个特定对象的位置和类别。通过学习基于图像的高层语义的动态系数的凸组合的方式，生成的动态查询更好地捕获不同图像中对象位置和类别的先验信息。利用我们的模态查询，一系列基于DETR的模型在多个任务中实现了一致且优秀的性能，包括物体检测，实例分割，全景分割和视频实例分割。

DQ-Det：基于Transformer的物体检测与分割的学习动态查询组合