Continual table semantic parsing aims to train a parser on a sequence of
tasks, where each task requires the parser to translate natural language into
SQL based on task-specific tables but only offers limited training examples.
Conventional methods tend to suffer from overfitting with limited supervision,
as well as catastrophic forgetting due to parameter updates. Despite recent
advancements that partially alleviate these issues through semi-supervised data
augmentation and retention of a few past examples, the performance is still
limited by the volume of unsupervised data and stored examples. To overcome
these challenges, this paper introduces a novel method integrating
\textit{parameter-efficient fine-tuning} (PEFT) and \textit{in-context tuning}
(ICT) for training a continual table semantic parser. Initially, we present a
task-adaptive PEFT framework capable of fully circumventing catastrophic
forgetting, which is achieved by freezing the pre-trained model backbone and
fine-tuning small-scale prompts. Building on this, we propose a teacher-student
framework-based solution. The teacher addresses the few-shot problem using ICT,
which procures contextual information by demonstrating a few training examples.
In turn, the student leverages the proposed PEFT framework to learn from the
teacher's output distribution, and subsequently compresses and saves the
contextual information to the prompts, eliminating the need to store any
training examples. Experimental evaluations on two benchmarks affirm the
superiority of our method over prevalent few-shot and continual learning
baselines across various metrics.

本文介绍了一个用于训练连续表语义解析器的新方法，该方法整合了参数高效微调（PEFT）和上下文微调（ICT），通过冻结预训练模型骨干和微调小规模提示来完全避免灾难性遗忘，并且通过教师 - 学生框架解决了少样本学习问题，通过 ICT 获得上下文信息来演示几个训练样本，然后学生利用 PEFT 框架从教师的输出分布中学习，并将上下文信息压缩保存到提示中，消除了存储任何训练样本的需要。在两个基准测试中进行的实验评估确认了我们方法在各种度量标准上相对于常见的少样本和连续学习基准的优越性。