Self-correction in text-to-SQL is the process of prompting large language
model (LLM) to revise its previously incorrectly generated SQL, and commonly
relies on manually crafted self-correction guidelines by human experts that are
not only labor-intensive to produce but also limited by the human ability in
identifying all potential error patterns in LLM responses. We introduce MAGIC,
a novel multi-agent method that automates the creation of the self-correction
guideline. MAGIC uses three specialized agents: a manager, a correction, and a
feedback agent. These agents collaborate on the failures of an LLM-based method
on the training set to iteratively generate and refine a self-correction
guideline tailored to LLM mistakes, mirroring human processes but without human
involvement. Our extensive experiments show that MAGIC's guideline outperforms
expert human's created ones. We empirically find out that the guideline
produced by MAGIC enhance the interpretability of the corrections made,
providing insights in analyzing the reason behind the failures and successes of
LLMs in self-correction. We make all agent interactions publicly available to
the research community, to foster further research in this area, offering a
synthetic dataset for future explorations into automatic self-correction
guideline generation.

MAGIC 是一种新的多代理方法，自动创建自纠正指南，通过在训练集上对基于 LLM 的方法的失败进行合作，迭代生成和改进适合 LLM 错误的自纠正指南，提高了纠正的可解释性，并提供了分析自纠正中 LLM 失败和成功原因的见解。

MAGIC：生成上下文文本转结构化查询语言的自我修正指南

MAGIC: Generating Self-Correction Guideline for In-Context Text-to-SQL

Large Language Models (LLMs) have demonstrated impressive and diverse
abilities that can benefit various domains, such as zero and few-shot
information extraction from clinical text without domain-specific training.
However, for the ICD coding task, they often hallucinate key details and
produce high recall but low precision results due to the high-dimensional and
skewed distribution of the ICD codes. Existing LLM-based methods fail to
account for the complex and dynamic interactions among the human agents
involved in coding, such as patients, physicians, and coders, and they lack
interpretability and reliability. In this paper, we present a novel multi-agent
method for ICD coding, which mimics the real-world coding process with five
agents: a patient agent, a physician agent, a coder agent, a reviewer agent,
and an adjuster agent. Each agent has a specific function and uses a LLM-based
model to perform it. We evaluate our method on the MIMIC-III dataset and show
that our proposed multi-agent coding framework substantially improves
performance on both common and rare codes compared to Zero-shot Chain of
Thought (CoT) prompting and self-consistency with CoT. The ablation study
confirms the proposed agent roles' efficacy. Our method also matches the
state-of-the-art ICD coding methods that require pre-training or fine-tuning,
in terms of coding accuracy, rare code accuracy, and explainability.

本文介绍了一种新颖的基于多智能体方法的 ICD 编码方法，模仿真实世界的编码过程，并使用基于大型语言模型的模型进行编码。通过在 MIMIC-III 数据集上进行评估，我们展示了我们提出的多智能体编码框架在常见和罕见代码方面相对于零样本 CoT 提示和 CoT 的自一致性有显著的性能提升。消融研究证实了提出的智能体角色的有效性。我们的方法在编码准确性、罕见代码准确性和可解释性方面也达到了需要预训练或微调的 ICD 编码方法的最新水平。