Monolingual data have been demonstrated to be helpful in improving
translation quality of both statistical machine translation (SMT) systems and
neural machine translation (NMT) systems, especially in resource-poor or domain
adaptation tasks where parallel data are not rich enough. In this paper, we
propose a novel approach to better leveraging monolingual data for neural
machine translation by jointly learning source-to-target and target-to-source
NMT models for a language pair with a joint EM optimization method. The
training process starts with two initial NMT models pre-trained on parallel
data for each direction, and these two models are iteratively updated by
incrementally decreasing translation losses on training data. In each iteration
step, both NMT models are first used to translate monolingual data from one
language to the other, forming pseudo-training data of the other NMT model.
Then two new NMT models are learnt from parallel data together with the pseudo
training data. Both NMT models are expected to be improved and better
pseudo-training data can be generated in next step. Experiment results on
Chinese-English and English-German translation tasks show that our approach can
simultaneously improve translation quality of source-to-target and
target-to-source models, significantly outperforming strong baseline systems
which are enhanced with monolingual data for model training including
back-translation.

本文提出了一种新方法，通过联合 EM 优化方法融合源语言和目标语言的神经机器翻译模型，以更好地利用单语数据来提高翻译质量，实验结果表明，相对于使用单语数据训练的强基线系统，该方法可以同时提高源到目标和目标到源模型的翻译质量。

利用单语数据进行神经机器翻译模型的联合训练

Joint Training for Neural Machine Translation Models with Monolingual  Data

Discriminative latent-variable models are typically learned using EM or
gradient-based optimization, which suffer from local optima. In this paper, we
develop a new computationally efficient and provably consistent estimator for a
mixture of linear regressions, a simple instance of a discriminative
latent-variable model. Our approach relies on a low-rank linear regression to
recover a symmetric tensor, which can be factorized into the parameters using a
tensor power method. We prove rates of convergence for our estimator and
provide an empirical evaluation illustrating its strengths relative to local
optimization (EM).

本文提出一种新的基于低秩线性回归和张量幂法的计算高效且具有保证一致性的鉴别性潜变量模型估计方法，该方法在混合线性回归模型中被验证其收敛性能优于局部优化 EM 算法。