Few-shot in-context learning (ICL) enables pre-trained language models to
perform a previously-unseen task without any gradient-based training by feeding
a small number of training examples as part of the input. ICL incurs
substantial computational, memory, and storage costs because it involves
processing all of the training examples every time a prediction is made.
Parameter-efficient fine-tuning (PEFT) (e.g. adapter modules, prompt tuning,
sparse update methods, etc.) offers an alternative paradigm where a small set
of parameters are trained to enable a model to perform the new task. In this
paper, we rigorously compare few-shot ICL and PEFT and demonstrate that the
latter offers better accuracy as well as dramatically lower computational
costs. Along the way, we introduce a new PEFT method called (IA)$^3$ that
scales activations by learned vectors, attaining stronger performance while
only introducing a relatively tiny amount of new parameters. We also propose a
simple recipe based on the T0 model called T-Few that can be applied to new
tasks without task-specific tuning or modifications. We validate the
effectiveness of T-Few on completely unseen tasks by applying it to the RAFT
benchmark, attaining super-human performance for the first time and
outperforming the state-of-the-art by 6% absolute. All of the code used in our
experiments is publicly available.

通过比较 Few-shot in-context learning 和 Parameter-efficient fine-tuning 的方法，我们证明 PEFT 具有更好的精度和极低的计算成本。我们提出了一种名为 (IA)$^3$ 的新的 PEFT 方法，并提出了一个基于 T0 模型的简单配方，称为 T-Few，在没有任务特定调整或修改的情况下可以应用于新任务。我们在 RAF 基准测试中验证了 T-Few 的有效性，首次达到了超人类表现，并超越了现有技术的 6％。

少样本参数高效微调优于上下文学习且更为经济

Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning

Prompt-based approaches are strong at few-shot learning. However, Perez et
al. (2021) have recently cast doubt on their performance because they had
difficulty getting good results in a "true" few-shot setting in which prompts
and hyperparameters cannot be tuned on a dev set. In view of this, we conduct
an extensive study of PET, a method that combines textual instructions with
example-based finetuning. We show that, if correctly configured, PET performs
strongly in a true few-shot setting, i.e., without a dev set. Crucial for this
strong performance is PET's ability to intelligently handle multiple prompts.
We then put our findings to a real-world test by running PET on RAFT, a
benchmark of tasks taken directly from realistic NLP applications for which no
labeled dev or test sets are available. PET achieves a new state of the art on
RAFT and performs close to non-expert humans for 7 out of 11 tasks. These
results demonstrate that prompt-based learners like PET excel at true few-shot
learning and underpin our belief that learning from instructions will play an
important role on the path towards human-like few-shot learning capabilities.

PET 方法结合文本说明与示例微调，在真实的 few-shot 设置实现强大的性能表现，在 RAFT 基准下取得了新的最高水平，并在 11 项任务中有 7 项接近于非专家人类表现。

真正的提示式几 - shot 学习 —— 从现实角度出发

True Few-Shot Learning with Prompts -- A Real-World Perspective

Large pre-trained language models have shown promise for few-shot learning,
completing text-based tasks given only a few task-specific examples. Will
models soon solve classification tasks that have so far been reserved for human
research assistants? Existing benchmarks are not designed to measure progress
in applied settings, and so don't directly answer this question. The RAFT
benchmark (Real-world Annotated Few-shot Tasks) focuses on naturally occurring
tasks and uses an evaluation setup that mirrors deployment. Baseline
evaluations on RAFT reveal areas current techniques struggle with: reasoning
over long texts and tasks with many classes. Human baselines show that some
classification tasks are difficult for non-expert humans, reflecting that
real-world value sometimes depends on domain expertise. Yet even non-expert
human baseline F1 scores exceed GPT-3 by an average of 0.11. The RAFT datasets
and leaderboard will track which model improvements translate into real-world
benefits at this https URL .

该研究论文讨论预训练的语言模型在少量数据情况下完成自然语言处理任务的表现，提出了一种新的用于测量模型在真实世界中应用的少量样本任务的基准测试 RAFT，这个基准测试展示了当前技术存在的困难和挑战，虽然一些分类任务对于普通人来说也很困难，但普通人在这些任务上的表现超过了 GPT-3 模型。