Large language models (LLMs), such as ChatGPT, are able to generate
human-like, fluent responses for many downstream tasks, e.g., task-oriented
dialog and question answering. However, applying LLMs to real-world,
mission-critical applications remains challenging mainly due to their tendency
to generate hallucinations and their inability to use external knowledge. This
paper proposes a LLM-Augmenter system, which augments a black-box LLM with a
set of plug-and-play modules. Our system makes the LLM generate responses
grounded in external knowledge, e.g., stored in task-specific databases. It
also iteratively revises LLM prompts to improve model responses using feedback
generated by utility functions, e.g., the factuality score of a LLM-generated
response. The effectiveness of LLM-Augmenter is empirically validated on two
types of scenarios, task-oriented dialog and open-domain question answering.
LLM-Augmenter significantly reduces ChatGPT's hallucinations without
sacrificing the fluency and informativeness of its responses. We make the
source code and models publicly available.

本研究提出了一种 LLm-Augmenter 系统，它通过使用存储在特定任务数据库中的外部知识和迭代地根据效用函数生成的反馈来改善 LLM 的提示，从而使其生成基于外部知识的响应，实现了应用到任务导向对话和问题回答的实际场景中，显著减少 ChatGPT 模型的幻觉。