We propose an unsupervised method to extract keywords and keyphrases from
texts based on a pre-trained language model (LM) and Shannon's information
maximization. Specifically, our method extracts phrases having the highest
conditional entropy under the LM. The resulting set of keyphrases turns out to
solve a relevant information-theoretic problem: if provided as side
information, it leads to the expected minimal binary code length in compressing
the text using the LM and an entropy encoder. Alternately, the resulting set is
an approximation via a causal LM to the set of phrases that minimize the
entropy of the text when conditioned upon it. Empirically, the method provides
results comparable to the most commonly used methods in various keyphrase
extraction benchmark challenges.

这篇研究论文介绍了一种无监督的方法，利用预训练的语言模型和香农的信息最大化从文本中提取关键词和关键短语。通过提取在语言模型下具有最高条件熵的短语，该方法能够解决与信息理论相关的问题，并在使用语言模型和熵编码器对文本进行压缩时，提供预期的最小二进制码长度。这种方法在各种关键短语提取竞赛中的结果与最常用的方法相当。