We present a novel method named Latent Semantic Imputation (LSI) to transfer
external knowledge into semantic space for enhancing word embedding. The method
integrates graph theory to extract the latent manifold structure of the
entities in the affinity space and leverages non-negative least squares with
standard simplex constraints and power iteration method to derive spectral
embeddings. It provides an effective and efficient approach to combining entity
representations defined in different Euclidean spaces. Specifically, our
approach generates and imputes reliable embedding vectors for low-frequency
words in the semantic space and benefits downstream language tasks that depend
on word embedding. We conduct comprehensive experiments on a carefully designed
classification problem and language modeling and demonstrate the superiority of
the enhanced embedding via LSI over several well-known benchmark embeddings. We
also confirm the consistency of the results under different parameter settings
of our method.

本研究提出了一种名为潜在语义填补（LSI）的方法，利用图论提取亲和性空间内实体的潜在流形结构，结合非负最小二乘法和幂迭代法，将外部知识转移入语义空间，生成并填补语义空间中低频词的可靠嵌入向量，优化词嵌入效果，进而提升下游自然语言处理任务的性能。实验验证了该方法在分类和语言模型任务中的显著优于多种常用词嵌入方法，且结果在不同参数下稳定一致。