Neural networks have become the state-of-the-art approach for machine
translation (MT) in many languages. While linguistically-motivated tokenization
techniques were shown to have significant effects on the performance of
statistical MT, it remains unclear if those techniques are well suited for
neural MT. In this paper, we systematically compare neural and statistical MT
models for Arabic-English translation on data preprecossed by various prominent
tokenization schemes. Furthermore, we consider a range of data and vocabulary
sizes and compare their effect on both approaches. Our empirical results show
that the best choice of tokenization scheme is largely based on the type of
model and the size of data. We also show that we can gain significant
improvements using a system selection that combines the output from neural and
statistical MT.

本文系统比较了优秀的神经网络与基于统计学的机器翻译模型在不同华丽的预处理令牌方案下的阿拉伯语 - 英语翻译数据，并考虑对于这两种方法数据和词汇量的大小对性能的影响。实证结果显示，最佳的语言学基础令牌化方案的选择在于模型类型和数据规模的大小，并且使用神经网络与统计学的机器翻译输出的系统选择可以获得显著的性能提升。

预处理对阿拉伯语 - 英语统计和神经机器翻译的影响

The Impact of Preprocessing on Arabic-English Statistical and Neural  Machine Translation

The goal of this work is to design a machine translation (MT) system for a
low-resource family of dialects, collectively known as Swiss German, which are
widely spoken in Switzerland but seldom written. We collected a significant
number of parallel written resources to start with, up to a total of about 60k
words. Moreover, we identified several other promising data sources for Swiss
German. Then, we designed and compared three strategies for normalizing Swiss
German input in order to address the regional diversity. We found that
character-based neural MT was the best solution for text normalization. In
combination with phrase-based statistical MT, our solution reached 36% BLEU
score when translating from the Bernese dialect. This value, however, decreases
as the testing data becomes more remote from the training one, geographically
and topically. These resources and normalization techniques are a first step
towards full MT of Swiss German dialects.

设计了一种机器翻译系统，使用字符级神经网络翻译文本，并结合基于短语的统计翻译，实现了对伯尔尼方言的 36% BLEU 得分的翻译。这些资源和规范化技术是通向瑞士德语方言全面机器翻译的第一步。