Text-to-Text Transfer Transformer (T5) has recently been considered for the
Grapheme-to-Phoneme (G2P) transduction. As a follow-up, a tokenizer-free
byte-level model based on T5 referred to as ByT5, recently gave promising
results on word-level G2P conversion by representing each input character with
its corresponding UTF-8 encoding. Although it is generally understood that
sentence-level or paragraph-level G2P can improve usability in real-world
applications as it is better suited to perform on heteronyms and linking sounds
between words, we find that using ByT5 for these scenarios is nontrivial. Since
ByT5 operates on the character level, it requires longer decoding steps, which
deteriorates the performance due to the exposure bias commonly observed in
auto-regressive generation models. This paper shows that the performance of
sentence-level and paragraph-level G2P can be improved by mitigating such
exposure bias using our proposed loss-based sampling method.

该研究展示了使用我们提出的基于损失的采样方法可以缓解自回归生成模型中的曝光偏差，从而改善句级别和段落级别的 G2P 转换性能。

缓解句级音素转换中的接触偏差

Mitigating the Exposure Bias in Sentence-Level Grapheme-to-Phoneme (G2P)  Transduction

This paper presents an automatic approach to creating taxonomies of technical
terms based on the Cooperative Patent Classification (CPC). The resulting
taxonomy contains about 170k nodes in 9 separate technological branches and is
freely available. We also show that a Text-to-Text Transfer Transformer (T5)
model can be fine-tuned to generate hypernyms and hyponyms with relatively high
precision, confirming the manually assessed quality of the resource. The T5
model opens the taxonomy to any new technological terms for which a hypernym
can be generated, thus making the resource updateable with new terms, an
essential feature for the constantly evolving field of technological
terminology.

本文提出了一种基于专利合作分类（CPC）的技术术语分类自动化方法，生成的分类包含 9 个独立技术分支中的约 170k 个节点，并且使用 Text-to-Text Transfer Transformer (T5) 模型 fine-tune 生成上下位词并得到相对较高的准确性，从而证明了此资源的质量。T5 模型可以打开新的技术术语的分类（根据上下位关系），因此这个资源可以更新，是技术术语领域不断发展的必要特性。

专利文本中用于上下位词检索的技术分类

Technological taxonomies for hypernym and hyponym retrieval in patent texts

The paper explores the relevance of the Text-To-Text Transfer Transformer
language model (T5) for Polish (plT5) to the task of intrinsic and extrinsic
keyword extraction from short text passages. The evaluation is carried out on
the new Polish Open Science Metadata Corpus (POSMAC), which is released with
this paper: a collection of 216,214 abstracts of scientific publications
compiled in the CURLICAT project. We compare the results obtained by four
different methods, i.e. plT5kw, extremeText, TermoPL, KeyBERT and conclude that
the plT5kw model yields particularly promising results for both frequent and
sparsely represented keywords. Furthermore, a plT5kw keyword generation model
trained on the POSMAC also seems to produce highly useful results in
cross-domain text labelling scenarios. We discuss the performance of the model
on news stories and phone-based dialog transcripts which represent text genres
and domains extrinsic to the dataset of scientific abstracts. Finally, we also
attempt to characterize the challenges of evaluating a text-to-text model on
both intrinsic and extrinsic keyword extraction.

探讨了 Text-To-Text Transfer Transformer (T5) 语言模型在波兰语 (plT5) 中对短文本段进行内在和外在关键词提取的相关性。通过对新的波兰开放科学元数据语料库（POSMAC）的评估，比较了四种不同方法得到的结果，其中 plT5kw 模型在频繁和稀疏表示关键词方面得到了特别有前途的结果。还讨论了模型在新闻和电话对话稿件的表现，并尝试在内在和外部关键词提取的情况下评价文本模型的挑战。

使用文本对文本转换变压器从短文本中提取关键词

Keyword Extraction from Short Texts with a Text-To-Text Transfer Transformer

The recent "Text-to-Text Transfer Transformer" (T5) leveraged a unified
text-to-text format and scale to attain state-of-the-art results on a wide
variety of English-language NLP tasks. In this paper, we introduce mT5, a
multilingual variant of T5 that was pre-trained on a new Common Crawl-based
dataset covering 101 languages. We detail the design and modified training of
mT5 and demonstrate its state-of-the-art performance on many multilingual
benchmarks. We also describe a simple technique to prevent "accidental
translation" in the zero-shot setting, where a generative model chooses to
(partially) translate its prediction into the wrong language. All of the code
and model checkpoints used in this work are publicly available.

本文介绍了 mT5，这是 T5 的多语言变体，基于新的基于 Common Crawl 的数据集进行预训练，涵盖 101 种语言，并展示了在许多多语言基准测试中的最新性能。我们还描述了一种简单的技术，用于在零 - shot 设置中防止 “意外翻译”。