语言模型概述：最新发展与展望

Mar, 2023

An Overview on Language Models: Recent Developments and Outlook

Chengwei Wei, Yun-Cheng Wang, Bin Wang, C. -C. Jay Kuo

TL;DR本文从语言单位、结构、训练方法、评估方法和应用等五个方面，介绍了传统语言模型和预训练语言模型，讨论了二者的关系和语言模型在预训练时代的未来发展方向。

Abstract

language modeling studies the probability distributions over strings of texts. It is one of the most fundamental tasks in natural language processing (nlp). It has been widely used in text generation, speech reco