NLP models excel on tasks with clean inputs, but are less accurate with noisy
inputs. In particular, character-level noise such as human-written typos and
adversarially-engineered realistic-looking misspellings often appears in text
and can easily trip up NLP models. Prior solutions to address character-level
noise often alter the content of the inputs (low fidelity), thus inadvertently
lowering model accuracy on clean inputs. We proposed FiRo, an approach to boost
NLP model performance on noisy inputs without sacrificing performance on clean
inputs. FiRo sanitizes the input text while preserving its fidelity by
inferring the noise-free form for each token in the input. FiRo uses
finite-context aggregation to obtain contextual embeddings which is then used
to find the noise-free form within a restricted output space. The output space
is restricted to a small cluster of probable candidates in order to predict the
noise-free tokens more accurately. Although the clusters are small, FiRo's
effective vocabulary (union of all clusters) can be scaled up to better
preserve the input content. Experimental results show NLP models that use FiRo
outperforming baselines on six classification tasks and one sequence labeling
task at various degrees of noise.

FiRo 是一种提高 NLP 模型性能的方法，可以清洗输入文本中的字符级噪声，同时保留其内容的真实性。实验结果表明，使用 FiRo 的 NLP 模型在不同程度的噪声下在六个分类任务和一个序列标记任务上优于基准模型。

自然语言处理模型面临嘈杂输入的有限环境索引化

Finite-context Indexing of Restricted Output Space for NLP Models Facing  Noisy Input

In this work, we induce character-level noise in various forms when
fine-tuning BERT to enable zero-shot cross-lingual transfer to unseen dialects
and languages. We fine-tune BERT on three sentence-level classification tasks
and evaluate our approach on an assortment of unseen dialects and languages. We
find that character-level noise can be an extremely effective agent of
cross-lingual transfer under certain conditions, while it is not as helpful in
others. Specifically, we explore these differences in terms of the nature of
the task and the relationships between source and target languages, finding
that introduction of character-level noise during fine-tuning is particularly
helpful when a task draws on surface level cues and the source-target
cross-lingual pair has a relatively high lexical overlap with shorter (i.e.,
less meaningful) unseen tokens on average.

本文介绍了一种方法，通过在 fine-tuning BERT 模型时引入不同形式的字符级噪音，实现了对不同方言和语言的零 - shot 跨语言迁移。作者在三个句子级分类任务上 fine-tune BERT，并在若干未知的方言和语言上进行了评估。他们发现，在特定条件下，字符级噪音可以成为跨语言转移的极其有效的催化剂。特别是当任务依赖表层提示且源 - 目标跨语言语言对具有相对较高的词汇重叠，并且平均具有较短（即含义较少）的未知令牌时，fine-tuning 过程中引入字符级噪音可以帮助更好地完成任务。

借助字符级噪声对 BERT 进行微调，实现零 - shot 迁移到方言和密切相关的语言

Fine-Tuning BERT with Character-Level Noise for Zero-Shot Transfer to  Dialects and Closely-Related Languages

Cross-lingual transfer between a high-resource language and its dialects or
closely related language varieties should be facilitated by their similarity.
However, current approaches that operate in the embedding space do not take
surface similarity into account. This work presents a simple yet effective
strategy to imrove cross-lingual transfer between closely related varieties. We
propose to augment the data of the high-resource source language with
character-level noise to make the model more robust towards spelling
variations. Our strategy shows consistent improvements over several languages
and tasks: Zero-shot transfer of POS tagging and topic identification between
language varieties from the Finnic, West and North Germanic, and Western
Romance language branches. Our work provides evidence for the usefulness of
simple surface-level noise in improving transfer between language varieties.

通过在高资源语言中增加字符级噪音，我们提出了一种简单而有效的策略，以增强密切相关的语言之间的跨语言转移，从而提高 POS 标记和主题识别等任务的零样本转移能力。