Knowledge base question answering (KBQA) is a critical yet challenging task
due to the vast number of entities within knowledge bases and the diversity of
natural language questions posed by users. Unfortunately, the performance of
most KBQA models tends to decline significantly in real-world scenarios where
high-quality annotated data is insufficient. To mitigate the burden associated
with manual annotation, we introduce FlexKBQA by utilizing Large Language
Models (LLMs) as program translators for addressing the challenges inherent in
the few-shot KBQA task. Specifically, FlexKBQA leverages automated algorithms
to sample diverse programs, such as SPARQL queries, from the knowledge base,
which are subsequently converted into natural language questions via LLMs. This
synthetic dataset facilitates training a specialized lightweight model for the
KB. Additionally, to reduce the barriers of distribution shift between
synthetic data and real user questions, FlexKBQA introduces an executionguided
self-training method to iterative leverage unlabeled user questions.
Furthermore, we explore harnessing the inherent reasoning capability of LLMs to
enhance the entire framework. Consequently, FlexKBQA delivers substantial
flexibility, encompassing data annotation, deployment, and being domain
agnostic. Through extensive experiments on GrailQA, WebQSP, and KQA Pro, we
observe that under the few-shot even the more challenging zero-shot scenarios,
FlexKBQA achieves impressive results with a few annotations, surpassing all
previous baselines and even approaching the performance of supervised models,
achieving a remarkable 93% performance relative to the fully-supervised models.
We posit that FlexKBQA represents a significant advancement towards exploring
better integration of large and lightweight models. The code is open-sourced.

FlexKBQA 是一种利用大型语言模型解决少样本知识库问答任务的方法，通过自动化算法从知识库中生成多样的程序，利用语言模型将其转化为自然语言问题。该方法还引入了执行引导自训练的策略来缓解合成数据和真实用户问题之间的分布差异，并利用大型语言模型的推理能力来增强整个框架。在 GrailQA、WebQSP 和 KQA Pro 上的实验证明，FlexKBQA 在少样本甚至零样本情景下，只需少量注释即可达到令人印象深刻的结果，超过了所有之前的基准，并且接近于监督模型的性能，相对于完全监督模型性能达到 93%。我们认为 FlexKBQA 在大型和轻量级模型的更好集成方面代表了重要进展。