Semitic morphologically-rich languages (MRLs) are characterized by extreme
word ambiguity. Because most vowels are omitted in standard texts, many of the
words are homographs with multiple possible analyses, each with a different
pronunciation and different morphosyntactic properties. This ambiguity goes
beyond word-sense disambiguation (WSD), and may include token segmentation into
multiple word units. Previous research on MRLs claimed that standardly trained
pre-trained language models (PLMs) based on word-pieces may not sufficiently
capture the internal structure of such tokens in order to distinguish between
these analyses. Taking Hebrew as a case study, we investigate the extent to
which Hebrew homographs can be disambiguated and analyzed using PLMs. We
evaluate all existing models for contextualized Hebrew embeddings on a novel
Hebrew homograph challenge sets that we deliver. Our empirical results
demonstrate that contemporary Hebrew contextualized embeddings outperform
non-contextualized embeddings; and that they are most effective for
disambiguating segmentation and morphosyntactic features, less so regarding
pure word-sense disambiguation. We show that these embeddings are more
effective when the number of word-piece splits is limited, and they are more
effective for 2-way and 3-way ambiguities than for 4-way ambiguity. We show
that the embeddings are equally effective for homographs of both balanced and
skewed distributions, whether calculated as masked or unmasked tokens. Finally,
we show that these embeddings are as effective for homograph disambiguation
with extensive supervised training as with a few-shot setup.

研究表明，最新的希伯来语情境嵌入模型在消除希伯来语同形异义词和分析词素、句法特征方面效果显著，尤其在限制分词数、2 种和 3 种歧义的情况下，对于平衡和不平衡分布的同形异义词同样有效，无论是有监督训练还是几次训练。

预训练上下文语言模型对希伯来同形字分析的区分能力

Do Pretrained Contextual Language Models Distinguish between Hebrew  Homograph Analyses?

Chinese short text matching is a fundamental task in natural language
processing. Existing approaches usually take Chinese characters or words as
input tokens. They have two limitations: 1) Some Chinese words are polysemous,
and semantic information is not fully utilized. 2) Some models suffer potential
issues caused by word segmentation. Here we introduce HowNet as an external
knowledge base and propose a Linguistic knowledge Enhanced graph Transformer
(LET) to deal with word ambiguity. Additionally, we adopt the word lattice
graph as input to maintain multi-granularity information. Our model is also
complementary to pre-trained language models. Experimental results on two
Chinese datasets show that our models outperform various typical text matching
approaches. Ablation study also indicates that both semantic information and
multi-granularity information are important for text matching modeling.

该研究提出了一种基于外部知识库 HowNet 和 Linguistic knowledge Enhanced graph Transformer（LET）来处理汉语短文本匹配中词语歧义性问题的模型，并采用单词格子图作为输入，取得了比其他典型文本匹配方法更好的结果。消融实验结果表明，语义信息和多粒度信息都对文本匹配建模至关重要。

LET: 基于语言知识增强的中文短文本匹配图卷积网络

LET: Linguistic Knowledge Enhanced Graph Transformer for Chinese Short  Text Matching

Recently proposed Skip-gram model is a powerful method for learning
high-dimensional word representations that capture rich semantic relationships
between words. However, Skip-gram as well as most prior work on learning word
representations does not take into account word ambiguity and maintain only
single representation per word. Although a number of Skip-gram modifications
were proposed to overcome this limitation and learn multi-prototype word
representations, they either require a known number of word meanings or learn
them using greedy heuristic approaches. In this paper we propose the Adaptive
Skip-gram model which is a nonparametric Bayesian extension of Skip-gram
capable to automatically learn the required number of representations for all
words at desired semantic resolution. We derive efficient online variational
learning algorithm for the model and empirically demonstrate its efficiency on
word-sense induction task.

本文介绍了一种自适应 Skip-gram 模型，能够自动学习所有单词在所需语义分辨率下所需的表示，并通过词义归纳任务证明了其效率。