Current approaches for fixing systematic problems in NLP models (e.g. regex patches, finetuning on more data) are either brittle, or labor-intensive and liable to shortcuts. In contrast, humans often provide corrections to each other through natural language. Taking inspiration from this, we explore natural language patches -- declarative statements that allow developers to provide corrective feedback at the right level of abstraction, either overriding the model (``if a review gives 2 stars, the sentiment is negative'') or providing additional information the model may lack (``if something is described as the bomb, then it is good''). We model the task of determining if a patch applies separately from the task of integrating patch information, and show that with a small amount of synthetic data, we can teach models to effectively use real patches on real data -- 1 to 7 patches improve accuracy by ~1-4 accuracy points on different slices of a sentiment analysis dataset, and F1 by 7 points on a relation extraction dataset. Finally, we show that finetuning on as many as 100 labeled examples may be needed to match the performance of a small set of language patches.

通过自然语言补丁，我们能够提供正确的抽象层次，从而为 NLP 模型提供更准确的校正反馈，此任务与整合补丁信息的任务可以分开，使用少量的合成数据来教导模型，能有效地使用实际数据上应用的补丁，使用 1 至 7 个补丁可以提高情感分析数据集上 ~1-4 的准确性点和关系提取数据集上 F1 7 点的精度，最后我们展示了即使有 100 个从标签数据示例微调模型，与一组小的语言补丁相比，也需要更多的时间。

使用自然语言补丁修复模型缺陷