Structured prediction tasks, like machine translation, involve learning
functions that map structured inputs to structured outputs. Recurrent Neural
Networks (RNNs) have historically been a popular choice for such tasks,
including in natural language processing (NLP) applications. However, training
RNNs using Maximum Likelihood Estimation (MLE) has its limitations, including
exposure bias and a mismatch between training and testing metrics. SEARNN,
based on the learning to search (L2S) framework, has been proposed as an
alternative to MLE for RNN training. This project explored the potential of
SEARNN to improve machine translation for low-resourced African languages -- a
challenging task characterized by limited training data availability and the
morphological complexity of the languages. Through experiments conducted on
translation for English to Igbo, French to \ewe, and French to \ghomala
directions, this project evaluated the efficacy of SEARNN over MLE in
addressing the unique challenges posed by these languages. With an average BLEU
score improvement of $5.4$\% over the MLE objective, we proved that SEARNN is
indeed a viable algorithm to effectively train RNNs on machine translation for
low-resourced languages.

通过对英语到伊博语、法语到埃维语以及法语到戈马拉语的翻译实验，本项目评估了 SEARNN 在解决低资源语言带来的独特挑战方面与 MLE 相比的功效，结果显示 SEARNN 在机器翻译低资源语言领域的训练具有显著的 BLEU 分数提升，证明了其作为可行算法的有效性。

超越 MLE：探索 SEARNN 用于低资源的神经机器翻译

Beyond MLE: Investigating SEARNN for Low-Resourced Neural Machine  Translation

We propose SEARNN, a novel training algorithm for recurrent neural networks
(RNNs) inspired by the "learning to search" (L2S) approach to structured
prediction. RNNs have been widely successful in structured prediction
applications such as machine translation or parsing, and are commonly trained
using maximum likelihood estimation (MLE). Unfortunately, this training loss is
not always an appropriate surrogate for the test error: by only maximizing the
ground truth probability, it fails to exploit the wealth of information offered
by structured losses. Further, it introduces discrepancies between training and
predicting (such as exposure bias) that may hurt test performance. Instead,
SEARNN leverages test-alike search space exploration to introduce global-local
losses that are closer to the test error. We first demonstrate improved
performance over MLE on two different tasks: OCR and spelling correction. Then,
we propose a subsampling strategy to enable SEARNN to scale to large vocabulary
sizes. This allows us to validate the benefits of our approach on a machine
translation task.

SEARNN 是一种基于 “学习搜索” 方法的递归神经网络（RNNs）训练算法，使用类似测试的搜索空间探索，引入了接近测试误差的全局局部损失，从而取得了复杂预测任务领域更好的表现。