Pre-trained masked language models successfully perform few-shot learning by
formulating downstream tasks as text infilling. However, as a strong
alternative in full-shot settings, discriminative pre-trained models like
ELECTRA do not fit into the paradigm. In this work, we adapt prompt-based
few-shot learning to ELECTRA and show that it outperforms masked language
models in a wide range of tasks. ELECTRA is pre-trained to distinguish if a
token is generated or original. We naturally extend that to prompt-based
few-shot learning by training to score the originality of the target options
without introducing new parameters. Our method can be easily adapted to tasks
involving multi-token predictions without extra computation overhead. Analysis
shows that ELECTRA learns distributions that align better with downstream
tasks.

本文提出了将 prompt-based few-shot learning 方法应用到 ELECTRA 上，表明该方法在各种任务中均胜过 masked language models，并显示 ELECTRA 学习到的分布与下游任务更加一致。

ELECTRA 激励：用判别式预训练模型进行少样本学习

Prompting ELECTRA: Few-Shot Learning with Discriminative Pre-Trained Models

Considerable advancements have been made in various NLP tasks based on the
impressive power of large language models (LLMs) and many NLP applications are
deployed in our daily lives. In this work, we challenge the capability of LLMs
with the new task of Ethical Quandary Generative Question Answering. Ethical
quandary questions are more challenging to address because multiple conflicting
answers may exist to a single quandary. We explore the current capability of
LLMs in providing an answer with a deliberative exchange of different
perspectives to an ethical quandary, in the approach of Socratic philosophy,
instead of providing a closed answer like an oracle. We propose a model that
searches for different ethical principles applicable to the ethical quandary
and generates an answer conditioned on the chosen principles through
prompt-based few-shot learning. We also discuss the remaining challenges and
ethical issues involved in this task and suggest the direction toward
developing responsible NLP systems by incorporating human values explicitly.

本文探讨语言模型在以苏格拉底哲学为思想基础的多重透视下反应伦理困境的能力，并提出一种基于情境学习的模型，利用伦理原则生成回答。

探索回答开放式伦理困境问题

Towards Answering Open-ended Ethical Quandary Questions

Learning to converse using only a few examples is a great challenge in
conversational AI. The current best conversational models, which are either
good chit-chatters (e.g., BlenderBot) or goal-oriented systems (e.g., MinTL),
are language models (LMs) fine-tuned on large conversational datasets. Training
these models is expensive, both in terms of computational resources and time,
and it is hard to keep them up to date with new conversational skills. A simple
yet unexplored solution is prompt-based few-shot learning (Brown et al. 2020)
which does not require gradient-based fine-tuning but instead uses a few
examples in the LM context as the only source of learning. In this paper, we
explore prompt-based few-shot learning in dialogue tasks. We benchmark LMs of
different sizes in nine response generation tasks, which include four
knowledge-grounded tasks, a task-oriented generations task, three open-chat
tasks, and controlled stylistic generation, and five conversational parsing
tasks, which include dialogue state tracking, graph path generation, persona
information extraction, document retrieval, and internet query generation. The
current largest released LM (GPT-J-6B) using prompt-based few-shot learning,
and thus requiring no training, achieves competitive performance to fully
trained state-of-the-art models. Moreover, we propose a novel prompt-based
few-shot classifier, that also does not require any fine-tuning, to select the
most appropriate prompt given a dialogue history. Finally, by combining the
power of prompt-based few-shot learning and a Skill Selector, we create an
end-to-end chatbot named the Few-Shot Bot (FSB), which automatically selects
the most appropriate conversational skill, queries different knowledge bases or
the internet, and uses the retrieved knowledge to generate a human-like
response, all using only few dialogue examples per skill.

本文探讨了通过基于提示的少样本学习在对话任务中的应用，通过对多种大小的语言模型进行测试，提出一种新的无需微调的提示分类器，并结合技能选择器创建了一种称为 Few-Shot Bot 的端到端聊天机器人，只需使用少量对话示例便可以完成知识检索并生成人类般自然的响应。