One of the recent trends in vision problems is to use natural language
captions to describe the objects of interest. This approach can overcome some
limitations of traditional methods that rely on bounding boxes or category
annotations. This paper introduces a novel paradigm for Multiple Object
Tracking called Type-to-Track, which allows users to track objects in videos by
typing natural language descriptions. We present a new dataset for that
Grounded Multiple Object Tracking task, called GroOT, that contains videos with
various types of objects and their corresponding textual captions describing
their appearance and action in detail. Additionally, we introduce two new
evaluation protocols and formulate evaluation metrics specifically for this
task. We develop a new efficient method that models a transformer-based
eMbed-ENcoDE-extRact framework (MENDER) using the third-order tensor
decomposition. The experiments in five scenarios show that our MENDER approach
outperforms another two-stage design in terms of accuracy and efficiency, up to
14.7% accuracy and 4$\times$ speed faster.

本文提出了一种使用自然语言描述对象并进行多目标跟踪的新范例，介绍了包含文字描述的视频数据集 GroOT，提出了用于这一任务的两个新的评估协议和评估度量，并采用基于 transformer 的 MENDER 方法开发了一种高效的模型，证明其在准确性和效率方面均超越了其他两阶段设计，跟踪精度提高了 14.7％，速度提升了 4 倍。