Text-based safety classifiers are widely used for content moderation and
increasingly to tune generative language model behavior - a topic of growing
concern for the safety of digital assistants and chatbots. However, different
policies require different classifiers, and safety policies themselves improve
from iteration and adaptation. This paper introduces and evaluates methods for
agile text classification, whereby classifiers are trained using small,
targeted datasets that can be quickly developed for a particular policy.
Experimenting with 7 datasets from three safety-related domains, comprising 15
annotation schemes, led to our key finding: prompt-tuning large language
models, like PaLM 62B, with a labeled dataset of as few as 80 examples can
achieve state-of-the-art performance. We argue that this enables a paradigm
shift for text classification, especially for models supporting safer online
discourse. Instead of collecting millions of examples to attempt to create
universal safety classifiers over months or years, classifiers could be tuned
using small datasets, created by individuals or small organizations, tailored
for specific use cases, and iterated on and adapted in the time-span of a day.

本文介绍并评估了一种针对特定策略快速开发分类器的敏捷文本分类方法，实验表明使用尽少至 80 个示例的标记数据集对大型语言模型（PaLM 62B）进行快速调整能够实现最先进的性能，并认为这种方法有助于支持更安全的在线话语模型