Natural language numbers are an example of compositional structures, where
larger numbers are composed of operations on smaller numbers. Given that
compositional reasoning is a key to natural language understanding, we propose
novel multilingual probing tasks tested on DistilBERT, XLM, and BERT to
investigate for evidence of compositional reasoning over numerical data in
various natural language number systems. By using both grammaticality judgment
and value comparison classification tasks in English, Japanese, Danish, and
French, we find evidence that the information encoded in these pretrained
models' embeddings is sufficient for grammaticality judgments but generally not
for value comparisons. We analyze possible reasons for this and discuss how our
tasks could be extended in further studies.

本研究提出基于 DistilBERT、XLM 和 BERT 的多语言探究任务，探讨自然语言数字系统中数值数据的组合推理证据，并发现这些预训练模型嵌入中编码的信息足以支持合法性判断，但一般不适用于值比较。