Sign language translation has historically been peripheral to mainstream machine translation research. In order to help converge the fields, we introduce FLEURS-ASL, an extension of the multiway parallel benchmarks FLORES (for text) and FLEURS (for speech) to support their first sign language (as video), American Sign Language, translated by 5 Certified Deaf Interpreters. FLEURS-ASL can be used to evaluate a variety of tasks -- primarily sentence- and discourse-level translation -- between ASL and 200 other languages as text, or 102 languages as speech. We provide baselines for tasks from ASL to English text using a unified modeling approach that incorporates timestamp tokens and previous text tokens in a 34-second context window, trained on random video clips from YouTube-ASL. This model meets or exceeds the performance of phrase-level baselines while supporting a multitude of new tasks. We also use FLEURS-ASL to show that multimodal frontier models have virtually no understanding of ASL, underscoring the importance of including sign languages in standard evaluation suites.

本研究解决了美国手语在主流机器翻译研究中的边缘化问题，通过引入FLEURS-ASL，将美国手语纳入多语言评估基准。该论文提出了一种统一建模的方法，并展示了新模型在ASL到英语文本的翻译中表现优于以往的基线，强调了在标准评估中纳入手语的重要性。

FLEURS-ASL：在大规模多语种多任务评估中纳入美国手语