Modern language models are capable of contextualizing words based on their
surrounding context. However, this capability is often compromised due to
semantic change that leads to words being used in new, unexpected contexts not
encountered during pre-training. In this paper, we model \textit{semantic
change} by studying the effect of unexpected contexts introduced by
\textit{lexical replacements}. We propose a \textit{replacement schema} where a
target word is substituted with lexical replacements of varying relatedness,
thus simulating different kinds of semantic change. Furthermore, we leverage
the replacement schema as a basis for a novel \textit{interpretable} model for
semantic change. We are also the first to evaluate the use of LLaMa for
semantic change detection.

我们通过研究由词汇替换引入的意外上下文的影响，建模了语义变化，并提出了一种替换模式作为可解释的语义变化模型，同时还首次评估了使用 LLaMa 进行语义变化检测的方法。

通过词语替换分析语义变化

Analyzing Semantic Change through Lexical Replacements

The pervasiveness of intra-utterance code-switching (CS) in spoken content
requires that speech recognition (ASR) systems handle mixed language. Designing
a CS-ASR system has many challenges, mainly due to data scarcity, grammatical
structure complexity, and domain mismatch. The most common method for
addressing CS is to train an ASR system with the available transcribed CS
speech, along with monolingual data. In this work, we propose a zero-shot
learning methodology for CS-ASR by augmenting the monolingual data with
artificially generating CS text. We based our approach on random lexical
replacements and Equivalence Constraint (EC) while exploiting aligned
translation pairs to generate random and grammatically valid CS content. Our
empirical results show a 65.5% relative reduction in language model perplexity,
and 7.7% in ASR WER on two ecologically valid CS test sets. The human
evaluation of the generated text using EC suggests that more than 80% is of
adequate quality.

本文基于随机词汇替换和等价约束，利用对齐翻译对生成随机合法的混合语言内容进行零样本学习，以解决跨语言语音识别中数据稀缺性、语法结构复杂性和领域匹配问题，实验结果显示，所提出的方法在两个生态有效的混合语言测试集上相对降低了 65.5% 的语言模型困惑度和 7.7% 的 ASR WER，而采用等价约束的人类评估表明，80% 以上的内容质量足够。