BriefGPT.xyz
Oct, 2020
mT5: 一种大规模多语言预训练文本到文本的转换器
mT5: A massively multilingual pre-trained text-to-text transformer
HTML
PDF
Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou...
TL;DR
本文介绍了mT5,这是T5的多语言变体,基于新的基于Common Crawl的数据集进行预训练,涵盖101种语言,并展示了在许多多语言基准测试中的最新性能。我们还描述了一种简单的技术,用于在零-shot设置中防止“意外翻译”。
Abstract
The recent "
text-to-text transfer transformer
" (T5) leveraged a unified text-to-text format and scale to attain state-of-the-art results on a wide variety of English-language NLP tasks. In this paper, we introduce
mt5
→