Sign language translation from video to spoken text presents unique
challenges owing to the distinct grammar, expression nuances, and high
variation of visual appearance across different speakers and contexts. The
intermediate gloss annotations of videos aim to guide the translation process.
In our work, we focus on {\em Gloss2Text} translation stage and propose several
advances by leveraging pre-trained large language models (LLMs), data
augmentation, and novel label-smoothing loss function exploiting gloss
translation ambiguities improving significantly the performance of
state-of-the-art approaches. Through extensive experiments and ablation studies
on the PHOENIX Weather 2014T dataset, our approach surpasses state-of-the-art
performance in {\em Gloss2Text} translation, indicating its efficacy in
addressing sign language translation and suggesting promising avenues for
future research and development.

我们的研究通过利用预训练的大型语言模型（LLMs）、数据增强和新的标签平滑损失函数，在中间注释的视频中，专注于 Gloss2Text 翻译阶段，并在 PHOENIX Weather 2014T 数据集上的广泛实验和消融研究中取得了显著的性能提升，为手语翻译领域的研究和发展提供了有效的方法和有前景的方向。

Gloss2Text: 使用 LLMs 和语义感知标签平滑的手语标记翻译

Gloss2Text: Sign Language Gloss translation using LLMs and Semantically  Aware Label Smoothing

While the progress of machine translation of written text has come far in the
past several years thanks to the increasing availability of parallel corpora
and corpora-based training technologies, automatic translation of spoken text
and dialogues remains challenging even for modern systems. In this paper, we
aim to boost the machine translation quality of conversational texts by
introducing a newly constructed Japanese-English business conversation parallel
corpus. A detailed analysis of the corpus is provided along with challenging
examples for automatic translation. We also experiment with adding the corpus
in a machine translation training scenario and show how the resulting system
benefits from its use.

本文旨在通过引入一个新构建的日英商务会话平行语料库，提高会话文本的机器翻译质量，并分析了该语料库，列举了自动翻译的具有挑战性的实例。同时，本文还试图将语料库添加到机器翻译训练中，并展示了基于该语料库训练的系统的优势。