Pre-trained language models (LMs) encode rich information about linguistic
structure but their knowledge about lexical polysemy remains unclear. We
propose a novel experimental setup for analysing this knowledge in LMs
specifically trained for different languages (English, French, Spanish and
Greek) and in multilingual BERT. We perform our analysis on datasets carefully
designed to reflect different sense distributions, and control for parameters
that are highly correlated with polysemy such as frequency and grammatical
category. We demonstrate that BERT-derived representations reflect words'
polysemy level and their partitionability into senses. Polysemy-related
information is more clearly present in English BERT embeddings, but models in
other languages also manage to establish relevant distinctions between words at
different polysemy levels. Our results contribute to a better understanding of
the knowledge encoded in contextualised representations and open up new avenues
for multilingual lexical semantics research.

该研究针对多语言预训练语言模型（LMs）的词汇多义性知识不清晰的问题，提出了一种新的实验方法，通过对反应词语多义性分布的数据集进行分析，控制与多义性高度相关的参数，证明 BERT-derived 的表示能够反映单词的多义水平以及它们拆分为不同义项。它揭示了 contextualized representations 中编码的知识并为多语言词汇语义研究开创了新的途径。