Sign Language Production (SLP) is the tough task of turning sign language
into sign videos. The main goal of SLP is to create these videos using a sign
gloss. In this research, we've developed a new method to make high-quality sign
videos without using human poses as a middle step. Our model works in two main
parts: first, it learns from a generator and the video's hidden features, and
next, it uses another model to understand the order of these hidden features.
To make this method even better for sign videos, we make several significant
improvements. (i) In the first stage, we take an improved 3D VQ-GAN to learn
downsampled latent representations. (ii) In the second stage, we introduce
sequence-to-sequence attention to better leverage conditional information.
(iii) The separated two-stage training discards the realistic visual semantic
of the latent codes in the second stage. To endow the latent sequences semantic
information, we extend the token-level autoregressive latent codes learning
with perceptual loss and reconstruction loss for the prior model with visual
perception. Compared with previous state-of-the-art approaches, our model
performs consistently better on two word-level sign language datasets, i.e.,
WLASL and NMFs-CSL.

经过改进，我们开发了一种新方法来制作高质量的手语视频，不需要使用人类姿势作为中间步骤。与之前的方法相比，我们的模型在两个手语数据集上表现更好。

使用隐藏动态变换器进行手语产生

Sign Language Production with Latent Motion Transformer

In this paper, we present SignAvatars, the first large-scale multi-prompt 3D
sign language (SL) motion dataset designed to bridge the communication gap for
hearing-impaired individuals. While there has been an exponentially growing
number of research regarding digital communication, the majority of existing
communication technologies primarily cater to spoken or written languages,
instead of SL, the essential communication method for hearing-impaired
communities. Existing SL datasets, dictionaries, and sign language production
(SLP) methods are typically limited to 2D as the annotating 3D models and
avatars for SL is usually an entirely manual and labor-intensive process
conducted by SL experts, often resulting in unnatural avatars. In response to
these challenges, we compile and curate the SignAvatars dataset, which
comprises 70,000 videos from 153 signers, totaling 8.34 million frames,
covering both isolated signs and continuous, co-articulated signs, with
multiple prompts including HamNoSys, spoken language, and words. To yield 3D
holistic annotations, including meshes and biomechanically-valid poses of body,
hands, and face, as well as 2D and 3D keypoints, we introduce an automated
annotation pipeline operating on our large corpus of SL videos. SignAvatars
facilitates various tasks such as 3D sign language recognition (SLR) and the
novel 3D SL production (SLP) from diverse inputs like text scripts, individual
words, and HamNoSys notation. Hence, to evaluate the potential of SignAvatars,
we further propose a unified benchmark of 3D SL holistic motion production. We
believe that this work is a significant step forward towards bringing the
digital world to the hearing-impaired communities. Our project page is at
this https URL

本文介绍了 SignAvatars 这个大规模的多提示 3D 手语运动数据集，旨在弥合听障个体之间的交流障碍；数据集涵盖包括隔离的手势和连贯的手势在内的 70,000 个视频，共计 8.34 百万帧，提供了自动化注释流水线以及 3D 手语的形态学注释，促进了 3D 手语识别和 3D 手语生成等任务的实施。

SignAvatars：大规模 3D 手语整体运动数据集与基准

SignAvatars: A Large-scale 3D Sign Language Holistic Motion Dataset and  Benchmark

Sign Language is the dominant yet non-primary form of communication language
used in the deaf and hearing-impaired community. To make an easy and mutual
communication between the hearing-impaired and the hearing communities,
building a robust system capable of translating the spoken language into sign
language and vice versa is fundamental. To this end, sign language recognition
and production are two necessary parts for making such a two-way system. Sign
language recognition and production need to cope with some critical challenges.
In this survey, we review recent advances in Sign Language Production (SLP) and
related areas using deep learning. This survey aims to briefly summarize recent
achievements in SLP, discussing their advantages, limitations, and future
directions of research.

本研究综述了深度学习在手语制作和相关领域上的最新进展以及所面临的挑战、优势、限制和未来研究方向。