Word embeddings and language models have transformed natural language processing (NLP) by facilitating the representation of linguistic elements in continuous vector spaces. This review visits foundational concepts such as the distributional hypothesis and contextual similarity, tracing the evolution from sparse representations like one-hot encoding to dense embeddings including Word2Vec, GloVe, and fastText. We examine both static and contextualized embeddings, underscoring advancements in models such as ELMo, BERT, and GPT and their adaptations for cross-lingual and personalized applications. The discussion extends to sentence and document embeddings, covering aggregation methods and generative topic models, along with the application of embeddings in multimodal domains, including vision, robotics, and cognitive science. Advanced topics such as model compression, interpretability, numerical encoding, and bias mitigation are analyzed, addressing both technical challenges and ethical implications. Additionally, we identify future research directions, emphasizing the need for scalable training techniques, enhanced interpretability, and robust grounding in non-textual modalities. By synthesizing current methodologies and emerging trends, this survey offers researchers and practitioners an in-depth resource to push the boundaries of embedding-based language models.

本文关注词嵌入和语言模型在自然语言处理中的重要性，探讨了从稀疏表示到密集嵌入的演变，尤其是在跨语言和个性化应用中的进展。作者提出了多模态嵌入的应用，分析了模型压缩与解释性等高级主题，并强调了未来研究方向，包括可扩展训练技术和非文本模式的稳健基础。该研究为推动嵌入基础语言模型的边界提供了深入的资源。

从词向量到多模态嵌入：大型语言模型的技术、应用与未来方向