BriefGPT.xyz
Aug, 2020
AMBERT:一个带有多粒度分词的预训练语言模型
AMBERT: A Pre-trained Language Model with Multi-Grained Tokenization
HTML
PDF
Xinsong Zhang, Hang Li
TL;DR
本文提出一种新的预训练语言模型 AMBERT,结合了精细的细粒度和粗粒度的分词方法,取得了比 BERT 更好的性能,特别是在汉语中表现更显著。此外,我们还开发了一种方法,用于提高 AMBERT 推理的效率,其仍然比 BERT 在同样计算成本下表现更好。
Abstract
pre-trained language models
such as
bert
have exhibited remarkable performances in many tasks in natural language understanding (NLU). The tokens in the models are usually fine-grained in the sense that for langu
→