The field of Sign Language Production (SLP) lacked a large-scale, pre-trained
model based on deep learning for continuous American Sign Language (ASL)
production in the past decade. This limitation hampers communication for all
individuals with disabilities relying on ASL. To address this issue, we
undertook the secondary development and utilization of How2Sign, one of the
largest publicly available ASL datasets. Despite its significance, prior
researchers in the field of sign language have not effectively employed this
corpus due to the intricacies involved in American Sign Language Production
(ASLP).
To conduct large-scale ASLP, we propose SignDiff based on the latest work in
related fields, which is a dual-condition diffusion pre-training model that can
generate human sign language speakers from a skeleton pose. SignDiff has a
novel Frame Reinforcement Network called FR-Net, similar to dense human pose
estimation work, which enhances the correspondence between text lexical symbols
and sign language dense pose frames reduce the occurrence of multiple fingers
in the diffusion model. In addition, our ASLP method proposes two new improved
modules and a new loss function to improve the accuracy and quality of sign
language skeletal posture and enhance the ability of the model to train on
large-scale data.
We propose the first baseline for ASL production and report the scores of
17.19 and 12.85 on BLEU-4 on the How2Sign dev/test sets. We also evaluated our
model on the previous mainstream dataset called PHOENIX14T, and the main
experiments achieved the results of SOTA. In addition, our image quality far
exceeds all previous results by 10 percentage points on the SSIM indicator.
Finally, we conducted ablation studies and qualitative evaluations for
discussion.

该研究论文介绍了一个基于深度学习的大规模美国手语（ASL）生成预训练模型，以解决对 ASL 依赖的残障人士之间的沟通障碍，并提出了用于 ASL 生成的基于条件扩散的预训练模型，同时改进了模型的准确性和质量，以及图像指标的优化。