Large language models (LLMs) have revolutionized the landscape of Natural
Language Processing systems, but are computationally expensive. To reduce the
cost without sacrificing performance, previous studies have explored various
approaches to harness the potential of Small Language Models (SLMs) as
cost-effective alternatives to their larger counterparts. Driven by findings
that SLMs and LLMs exhibit complementary strengths in a structured knowledge
extraction task, this work presents a novel SLM/LLM routing framework designed
to improve computational efficiency and enhance task performance. First,
exemplar pools are created to represent the types of contexts where each LM
provides a more reliable answer, leveraging a sentence embedding fine-tuned so
that context similarity is close to dialogue state similarity. Then, during
inference, the k-nearest exemplars to the testing instance are retrieved, and
the instance is routed according to majority vote. In dialogue state tracking
tasks, the proposed routing framework enhances performance substantially
compared to relying solely on LLMs, while reducing the computational costs by
over 50%.

通过创建示例池来代表每种语言模型较可靠回答的上下文类型，并利用经过微调的句子嵌入使上下文相似性接近对话状态相似性，本研究提出了一种新颖的 SLM/LLM 路由框架，旨在提高计算效率并增强任务性能，在对话状态跟踪任务中，相较于仅依赖 LLMs，所提出的路由框架显著提高性能，同时减少计算成本超过 50%。