BriefGPT.xyz
Apr, 2020
用语义类别探测量化单词表示的情境化
Quantifying the Contextualization of Word Representations with Semantic Class Probing
HTML
PDF
Mengjie Zhao, Philipp Dufter, Yadollah Yaghoobzadeh, Hinrich Schütze
TL;DR
通过研究BERT中的单词上下文含义的量化，我们展示了预训练语言模型如何解释单词在上下文中的重要性，结果显示顶层语义表示对于任务的推断非常准确，底层语义表示更具可转移性。
Abstract
pretrained language models
have achieved a new state of the art on many NLP tasks, but there are still many open questions about how and why they work so well. We investigate the
contextualization
of words in
→