This paper describes the University of Sydney& JD's joint submission of the
IWSLT 2021 low resource speech translation task. We participated in the
Swahili-English direction and got the best scareBLEU (25.3) score among all the
participants. Our constrained system is based on a pipeline framework, i.e. ASR
and NMT. We trained our models with the officially provided ASR and MT
datasets. The ASR system is based on the open-sourced tool Kaldi and this work
mainly explores how to make the most of the NMT models. To reduce the
punctuation errors generated by the ASR model, we employ our previous work
SlotRefine to train a punctuation correction model. To achieve better
translation performance, we explored the most recent effective strategies,
including back translation, knowledge distillation, multi-feature reranking and
transductive finetuning. For model structure, we tried auto-regressive and
non-autoregressive models, respectively. In addition, we proposed two novel
pre-train approaches, i.e. \textit{de-noising training} and
\textit{bidirectional training} to fully exploit the data. Extensive
experiments show that adding the above techniques consistently improves the
BLEU scores, and the final submission system outperforms the baseline
(Transformer ensemble model trained with the original parallel data) by
approximately 10.8 BLEU score, achieving the SOTA performance.

描述了悉尼大学＆JD 在 IWSLT 2021 低资源语音翻译任务中的联合提交。我们参加了斯瓦希里语 - 英语方向，并在所有参与者中获得了最佳的分数，使用基于管道框架的 ASR 和 NMT。我们采用了多种技术方法，包括基于知识蒸馏的后向转换、多特征重排和传导调整。在模型结构方面，我们分别尝试了自回归模型和非自回归模型，还提出了两种新颖的预训练方法，即去噪训练和双向训练，取得了 SOTA 的性能。