Topic models are frequently used in machine learning owing to their high
interpretability and modular structure. However, extending a topic model to
include a supervisory signal, to incorporate pre-trained word embedding vectors
and to include a nonlinear output function is not an easy task because one has
to resort to a highly intricate approximate inference procedure. The present
paper shows that topic modeling with pre-trained word embedding vectors can be
viewed as implementing a neighborhood aggregation algorithm where messages are
passed through a network defined over words. From the network view of topic
models, nodes correspond to words in a document and edges correspond to either
a relationship describing co-occurring words in a document or a relationship
describing the same word in the corpus. The network view allows us to extend
the model to include supervisory signals, incorporate pre-trained word
embedding vectors and include a nonlinear output function in a simple manner.
In experiments, we show that our approach outperforms the state-of-the-art
supervised Latent Dirichlet Allocation implementation in terms of held-out
document classification tasks.

本文利用网络视角研究主题模型，将节点与词语，边与文档内出现的词语之间的关系相对应，旨在探究如何将真实信号融入其模型中，并改善模型分类表现。

通过邻域聚合学习主题模型

Learning Topic Models by Neighborhood Aggregation

We first observe a potential weakness of continuous vector representations of
symbols in neural machine translation. That is, the continuous vector
representation, or a word embedding vector, of a symbol encodes multiple
dimensions of similarity, equivalent to encoding more than one meaning of the
word. This has the consequence that the encoder and decoder recurrent networks
in neural machine translation need to spend substantial amount of their
capacity in disambiguating source and target words based on the context which
is defined by a source sentence. Based on this observation, in this paper we
propose to contextualize the word embedding vectors using a nonlinear
bag-of-words representation of the source sentence. Additionally, we propose to
represent special tokens (such as numbers, proper nouns and acronyms) with
typed symbols to facilitate translating those words that are not well-suited to
be translated via continuous vectors. The experiments on En-Fr and En-De reveal
that the proposed approaches of contextualization and symbolization improves
the translation quality of neural machine translation systems significantly.

本文首次观察了神经机器翻译中不断向量表示的潜在弱点，基于此提出了利用非线性词袋表示法和类型化符号来提高对源语言和目标语言的区分度，从而显著提高了神经机器翻译系统的翻译质量。