The way the words are used evolves through time, mirroring cultural or
technological evolution of society. Semantic change detection is the task of
detecting and analysing word evolution in textual data, even in short periods
of time. In this paper we focus on a new set of methods relying on
contextualised embeddings, a type of semantic modelling that revolutionised the
NLP field recently. We leverage the ability of the transformer-based BERT model
to generate contextualised embeddings capable of detecting semantic change of
words across time. Several approaches are compared in a common setting in order
to establish strengths and weaknesses for each of them. We also propose several
ideas for improvements, managing to drastically improve the performance of
existing approaches.

本文旨在研究语义演变在文本数据中的检测与分析方法，特别是基于上下文嵌入的新方法，通过对比多个方法的表现以及提出相关改进策略，显著提高了现有方法的性能。

在词使用中捕获进化：只需增加更多聚类吗？

Capturing Evolution in Word Usage: Just Add More Clusters?

Word evolution refers to the changing meanings and associations of words
throughout time, as a byproduct of human language evolution. By studying word
evolution, we can infer social trends and language constructs over different
periods of human history. However, traditional techniques such as word
representation learning do not adequately capture the evolving language
structure and vocabulary. In this paper, we develop a dynamic statistical model
to learn time-aware word vector representation. We propose a model that
simultaneously learns time-aware embeddings and solves the resulting "alignment
problem". This model is trained on a crawled NYTimes dataset. Additionally, we
develop multiple intuitive evaluation strategies of temporal word embeddings.
Our qualitative and quantitative tests indicate that our method not only
reliably captures this evolution over time, but also consistently outperforms
state-of-the-art temporal embedding approaches on both semantic accuracy and
alignment quality.

该研究通过发展一种动态统计模型学习时态感知的词向量表示，并实现了 “对齐问题” 的解决，从而可以可靠地捕捉时间上的语言演变，并在语义准确性和对齐质量方面持续优于现有的时间嵌入方法。

动态词向量用于演化中的语义发现

Dynamic Word Embeddings for Evolving Semantic Discovery

We analyze the dynamic properties of 10^7 words recorded in English, Spanish
and Hebrew over the period 1800--2008 in order to gain insight into the
coevolution of language and culture. We report language independent patterns
useful as benchmarks for theoretical models of language evolution. A
significantly decreasing (increasing) trend in the birth (death) rate of words
indicates a recent shift in the selection laws governing word use. For new
words, we observe a peak in the growth-rate fluctuations around 40 years after
introduction, consistent with the typical entry time into standard dictionaries
and the human generational timescale. Pronounced changes in the dynamics of
language during periods of war shows that word correlations, occurring across
time and between words, are largely influenced by coevolutionary social,
technological, and political factors. We quantify cultural memory by analyzing
the long-term correlations in the use of individual words using detrended
fluctuation analysis.

通过分析英语、西班牙语和希伯来语记录的一千万个单词的动态特性，以期了解语言和文化的共同进化，我们报告了有用的语言独立模式，作为语言进化理论模型的基准。