BriefGPT.xyz
Oct, 2020
基于Transformer的语言模型的多语言数字理解探究
Probing for Multilingual Numerical Understanding in Transformer-Based Language Models
HTML
PDF
Devin Johnson, Denise Mak, Drew Barker, Lexi Loessberg-Zahl
TL;DR
本研究提出基于DistilBERT、XLM和BERT的多语言探究任务，探讨自然语言数字系统中数值数据的组合推理证据，并发现这些预训练模型嵌入中编码的信息足以支持合法性判断，但一般不适用于值比较。
Abstract
Natural language numbers are an example of
compositional structures
, where larger numbers are composed of operations on smaller numbers. Given that compositional reasoning is a key to
natural language understanding
→