Autonomous vehicle (AV) systems rely on robust perception models as a
cornerstone of safety assurance. However, objects encountered on the road
exhibit a long-tailed distribution, with rare or unseen categories posing
challenges to a deployed perception model. This necessitates an expensive
process of continuously curating and annotating data with significant human
effort. We propose to leverage recent advances in vision-language and large
language models to design an Automatic Data Engine (AIDE) that automatically
identifies issues, efficiently curates data, improves the model through
auto-labeling, and verifies the model through generation of diverse scenarios.
This process operates iteratively, allowing for continuous self-improvement of
the model. We further establish a benchmark for open-world detection on AV
datasets to comprehensively evaluate various learning paradigms, demonstrating
our method's superior performance at a reduced cost.

我们提出使用视觉语言和大型语言模型的最新进展设计自动数据引擎（AIDE），以自动识别问题、高效筛选数据、通过自动标注改进模型，并通过生成多样化场景验证模型，从而实现模型的持续自我改进。我们还在自动驾驶数据集上建立了一个开放世界检测基准，全面评估各种学习范式，并展示了我们方法在减少成本的基础上具有卓越性能。