We describe the JD Explore Academy's submission of the WMT 2022 shared
general translation task. We participated in all high-resource tracks and one
medium-resource track, including Chinese-English, German-English,
Czech-English, Russian-English, and Japanese-English. We push the limit of our
previous work -- bidirectional training for translation by scaling up two main
factors, i.e. language pairs and model sizes, namely the \textbf{Vega-MT}
system. As for language pairs, we scale the "bidirectional" up to the
"multidirectional" settings, covering all participating languages, to exploit
the common knowledge across languages, and transfer them to the downstream
bilingual tasks. As for model sizes, we scale the Transformer-Big up to the
extremely large model that owns nearly 4.7 Billion parameters, to fully enhance
the model capacity for our Vega-MT. Also, we adopt the data augmentation
strategies, e.g. cycle translation for monolingual data, and bidirectional
self-training for bilingual and monolingual data, to comprehensively exploit
the bilingual and monolingual data. To adapt our Vega-MT to the general domain
test set, generalization tuning is designed. Based on the official automatic
scores of constrained systems, in terms of the sacreBLEU shown in Figure-1, we
got the 1st place on {Zh-En (33.5), En-Zh (49.7), De-En (33.7), En-De (37.8),
Cs-En (54.9), En-Cs (41.4) and En-Ru (32.7)}, 2nd place on {Ru-En (45.1) and
Ja-En (25.6)}, and 3rd place on {En-Ja(41.5)}, respectively; W.R.T the COMET,
we got the 1st place on {Zh-En (45.1), En-Zh (61.7), De-En (58.0), En-De
(63.2), Cs-En (74.7), Ru-En (64.9), En-Ru (69.6) and En-Ja (65.1)}, 2nd place
on {En-Cs (95.3) and Ja-En (40.6)}, respectively. Models will be released to
facilitate the MT community through GitHub and OmniForce Platform.

通过多向设置和数据增强等方式，我们的 Vega-MT 在多语种翻译任务中取得了第一名，并将模型释放到 GitHub 和 OmniForce 平台进行进一步使用。

Vega-MT: WMT22 京东探索学院翻译系统

Vega-MT: The JD Explore Academy Translation System for WMT22

We present a simple and effective pretraining strategy -- bidirectional
training (BiT) for neural machine translation. Specifically, we bidirectionally
update the model parameters at the early stage and then tune the model
normally. To achieve bidirectional updating, we simply reconstruct the training
samples from "src$\rightarrow$tgt" to "src+tgt$\rightarrow$tgt+src" without any
complicated model modifications. Notably, our approach does not increase any
parameters or training steps, requiring the parallel data merely. Experimental
results show that BiT pushes the SOTA neural machine translation performance
across 15 translation tasks on 8 language pairs (data sizes range from 160K to
38M) significantly higher. Encouragingly, our proposed model can complement
existing data manipulation strategies, i.e. back translation, data
distillation, and data diversification. Extensive analyses show that our
approach functions as a novel bilingual code-switcher, obtaining better
bilingual alignment.

通过双向的预训练策略，将神经机器翻译模型从'src -> tgt' 朝向'src + tgt -> tgt + src' 方向进行更新，成功地提升了神经机器翻译在 15 项任务上的表现，具有更好的双语对齐。