Referring multi-object tracking (RMOT) aims at detecting and tracking
multiple objects following human instruction represented by a natural language
expression. Existing RMOT benchmarks are usually formulated through manual
annotations, integrated with static regulations. This approach results in a
dearth of notable diversity and a constrained scope of implementation. In this
work, our key idea is to bootstrap the task of referring multi-object tracking
by introducing discriminative language words as much as possible. In specific,
we first develop Refer-KITTI into a large-scale dataset, named Refer-KITTI-V2.
It starts with 2,719 manual annotations, addressing the issue of class
imbalance and introducing more keywords to make it closer to real-world
scenarios compared to Refer-KITTI. They are further expanded to a total of
9,758 annotations by prompting large language models, which create 617
different words, surpassing previous RMOT benchmarks. In addition, the
end-to-end framework in RMOT is also bootstrapped by a simple yet elegant
temporal advancement strategy, which achieves better performance than previous
approaches. The source code and dataset is available at
this https URL

通过引入具有鉴别力的语言词汇，我们首先将 Refer-KITTI 转化为大规模数据集 Refer-KITTI-V2，解决了类别不平衡问题，并通过大型语言模型推进策略扩展了数据集，包含了 617 个不同的关键词，超过了之前 RMOT 基准。此外，RMOT 的端到端框架还通过简单而优雅的时间推进策略进行了优化，表现优于之前的方法。