The demand for the retrieval of complex scene data in autonomous driving is
increasing, especially as passenger vehicles have been equipped with the
ability to navigate urban settings, with the imperative to address long-tail
scenarios. Meanwhile, under the pre-existing two dimensional image retrieval
method, some problems may arise with scene retrieval, such as lack of global
feature representation and subpar text retrieval ability. To address these
issues, we have proposed \textbf{BEV-CLIP}, the first multimodal Bird's-Eye
View(BEV) retrieval methodology that utilizes descriptive text as an input to
retrieve corresponding scenes. This methodology applies the semantic feature
extraction abilities of a large language model (LLM) to facilitate zero-shot
retrieval of extensive text descriptions, and incorporates semi-structured
information from a knowledge graph to improve the semantic richness and variety
of the language embedding. Our experiments result in 87.66% accuracy on
NuScenes dataset in text-to-BEV feature retrieval. The demonstrated cases in
our paper support that our retrieval method is also indicated to be effective
in identifying certain long-tail corner scenes.

我们提出了第一种多模式 Bird's-Eye View（BEV）检索方法 BEV-CLIP，它利用描述性文本作为输入来检索相应的场景，通过利用大型语言模型（LLM）的语义特征提取能力来实现零样本检索，并结合来自知识图的半结构化信息来提高语义丰富性和嵌入的多样性，我们的实验结果在 NuScenes 数据集的文本到 BEV 特征检索上达到了 87.66% 的准确率，我们论文中的示例案例支持我们的检索方法也可以有效识别某些长尾场景。