Our proposed method, ReSeTOX (REdo SEarch if TOXic), addresses the issue of
Neural Machine Translation (NMT) generating translation outputs that contain
toxic words not present in the input. The objective is to mitigate the
introduction of toxic language without the need for re-training. In the case of
identified added toxicity during the inference process, ReSeTOX dynamically
adjusts the key-value self-attention weights and re-evaluates the beam search
hypotheses. Experimental results demonstrate that ReSeTOX achieves a remarkable
57% reduction in added toxicity while maintaining an average translation
quality of 99.5% across 164 languages.

提出了一种名为 ReSeTOX 的方法，用于处理神经机器翻译（NMT）生成的输出中存在有害词的问题，目的是在无需重新训练的情况下减少有害语言的引入。经过实验结果表明，ReSeTOX 在保持平均翻译质量为 99.5％的情况下，在 164 种语言中实现了 57％的有害性降低。

ReSeTOX：重新学习注意力权重以减轻机器翻译中的毒性

ReSeTOX: Re-learning attention weights for toxicity mitigation in  machine translation

Transformer models have achieved promising results on natural language
processing (NLP) tasks including extractive question answering (QA). Common
Transformer encoders used in NLP tasks process the hidden states of all input
tokens in the context paragraph throughout all layers. However, different from
other tasks such as sequence classification, answering the raised question does
not necessarily need all the tokens in the context paragraph. Following this
motivation, we propose Block-skim, which learns to skim unnecessary context in
higher hidden layers to improve and accelerate the Transformer performance. The
key idea of Block-Skim is to identify the context that must be further
processed and those that could be safely discarded early on during inference.
Critically, we find that such information could be sufficiently derived from
the self-attention weights inside the Transformer model. We further prune the
hidden states corresponding to the unnecessary positions early in lower layers,
achieving significant inference-time speedup. To our surprise, we observe that
models pruned in this way outperform their full-size counterparts. Block-Skim
improves QA models' accuracy on different datasets and achieves 3 times speedup
on BERT-base model.

通过 Block-Skim 的方法，即从 Transformer 模型的内部权重中识别出需要进一步处理和可以尽早舍弃的上下文信息，从而明显提高了 QA 模型的准确性并实现了 3 倍加速。