Video Referring Expression Comprehension (REC) aims to localize a target
object in videos based on the queried natural language. Recent improvements in
video REC have been made using Transformer-based methods with learnable
queries. However, we contend that this naive query design is not ideal given
the open-world nature of video REC brought by text supervision. With numerous
potential semantic categories, relying on only a few slow-updated queries is
insufficient to characterize them. Our solution to this problem is to create
dynamic queries that are conditioned on both the input video and language to
model the diverse objects referred to. Specifically, we place a fixed number of
learnable bounding boxes throughout the frame and use corresponding region
features to provide prior information. Also, we noticed that current query
features overlook the importance of cross-modal alignment. To address this, we
align specific phrases in the sentence with semantically relevant visual areas,
annotating them in existing video datasets (VID-Sentence and VidSTG). By
incorporating these two designs, our proposed model (called ConFormer)
outperforms other models on widely benchmarked datasets. For example, in the
testing split of VID-Sentence dataset, ConFormer achieves 8.75% absolute
improvement on Accu.@0.6 compared to the previous state-of-the-art model.

使用动态查询和跨模态对齐，基于 Transformer 方法的 ConFormer 模型在视频参考表达理解中表现出色，实现了对目标对象的定位。

视频参照表达理解中的基于内容条件查询的 Transformer

Video Referring Expression Comprehension via Transformer with  Content-conditioned Query

3D object detection is crucial for applications like autonomous driving and
robotics. While query-based 3D object detection for BEV (Bird's Eye View)
images has seen significant advancements, most existing methods follows the
paradigm of static query. Such paradigm is incapable of adapting to complex
spatial-temporal relationships in the scene. To solve this problem, we
introduce a new paradigm in DynamicBEV, a novel approach that employs dynamic
queries for BEV-based 3D object detection. In contrast to static queries, the
proposed dynamic queries exploit K-means clustering and Top-K Attention in a
creative way to aggregate information more effectively from both local and
distant feature, which enable DynamicBEV to adapt iteratively to complex
scenes. To further boost efficiency, DynamicBEV incorporates a Lightweight
Temporal Fusion Module (LTFM), designed for efficient temporal context
integration with a significant computation reduction. Additionally, a
custom-designed Diversity Loss ensures a balanced feature representation across
scenarios. Extensive experiments on the nuScenes dataset validate the
effectiveness of DynamicBEV, establishing a new state-of-the-art and heralding
a paradigm-level breakthrough in query-based BEV object detection.

动态 BEV（Bird's Eye View）是一种新的查询方法，利用动态查询和动态场景下的聚类以及注意力机制，有效地从局部和远距离特征中聚合信息，实现对复杂场景的迭代适应，进而提高三维物体检测的效率和性能，从而在查询式 BEV 物体检测领域取得了划时代的突破。