Despite existing pioneering works on sign language translation (SLT), there
is a non-trivial obstacle, i.e., the limited quantity of parallel sign-text
data. To tackle this parallel data bottleneck, we propose a sign
back-translation (SignBT) approach, which incorporates massive spoken language
texts into SLT training. With a text-to-gloss translation model, we first
back-translate the monolingual text to its gloss sequence. Then, the paired
sign sequence is generated by splicing pieces from an estimated gloss-to-sign
bank at the feature level. Finally, the synthetic parallel data serves as a
strong supplement for the end-to-end training of the encoder-decoder SLT
framework.
To promote the SLT research, we further contribute CSL-Daily, a large-scale
continuous SLT dataset. It provides both spoken language translations and
gloss-level annotations. The topic revolves around people's daily lives (e.g.,
travel, shopping, medical care), the most likely SLT application scenario.
Extensive experimental results and analysis of SLT methods are reported on
CSL-Daily. With the proposed sign back-translation method, we obtain a
substantial improvement over previous state-of-the-art SLT methods.

本研究提出了一种标志性的回译（SignBT）方法，利用大规模口语文本来协助手语翻译（SLT）的训练。此外，我们还发布了一个大规模的连续 SLT 数据集 CSL-Daily，提供口语翻译和注释。通过提出的回译方法，我们实现了对先前 SLT 方法的显着改善。