Since American Sign Language (ASL) has no standard written form, Deaf signers
frequently share videos in order to communicate in their native language.
However, since both hands and face convey critical linguistic information in
signed languages, sign language videos cannot preserve signer privacy. While
signers have expressed interest, for a variety of applications, in sign
language video anonymization that would effectively preserve linguistic
content, attempts to develop such technology have had limited success, given
the complexity of hand movements and facial expressions. Existing approaches
rely predominantly on precise pose estimations of the signer in video footage
and often require sign language video datasets for training. These requirements
prevent them from processing videos 'in the wild,' in part because of the
limited diversity present in current sign language video datasets. To address
these limitations, our research introduces DiffSLVA, a novel methodology that
utilizes pre-trained large-scale diffusion models for zero-shot text-guided
sign language video anonymization. We incorporate ControlNet, which leverages
low-level image features such as HED (Holistically-Nested Edge Detection)
edges, to circumvent the need for pose estimation. Additionally, we develop a
specialized module dedicated to capturing facial expressions, which are
critical for conveying essential linguistic information in signed languages. We
then combine the above methods to achieve anonymization that better preserves
the essential linguistic content of the original signer. This innovative
methodology makes possible, for the first time, sign language video
anonymization that could be used for real-world applications, which would offer
significant benefits to the Deaf and Hard-of-Hearing communities. We
demonstrate the effectiveness of our approach with a series of signer
anonymization experiments.

通过 DiffSLVA 方法，利用预训练的大规模扩散模型和低级图像特征来进行零样本文本引导的手语视频匿名化，解决了手势识别的复杂性和对数据集的依赖性的问题，实现了更好地保护原始签名者的重要语言内容。

DiffSLVA：利用扩散模型实现手语视频匿名化

DiffSLVA: Harnessing Diffusion Models for Sign Language Video  Anonymization

This article presents an original method for Text-to-Sign Translation. It
compensates data scarcity using a domain-specific parallel corpus of alignments
between text and hierarchical formal descriptions of Sign Language videos in
AZee. Based on the detection of similarities present in the source text, the
proposed algorithm recursively exploits matches and substitutions of aligned
segments to build multiple candidate translations for a novel statement. This
helps preserving Sign Language structures as much as possible before falling
back on literal translations too quickly, in a generative way. The resulting
translations are in the form of AZee expressions, designed to be used as input
to avatar synthesis systems. We present a test set tailored to showcase its
potential for expressiveness and generation of idiomatic target language, and
observed limitations. This work finally opens prospects on how to evaluate
translation and linguistic aspects, such as accuracy and grammatical fluency.

本文提出了一种基于 AZee 的文本到手语翻译的算法，利用特定领域的平行语料库来解决数据短缺的问题，通过检测源文本中存在的相似性，递归地利用匹配和对齐段的替换来建立一个句子的多个候选翻译，这有助于以一种生成的方式尽可能保留手语结构。最终的翻译以 AZee 表达式的形式呈现，旨在作为输入到 Avatar 合成系统中。我们提供了一个测试集来展示其表现，并观察到了一些局限性。这项工作最终开启了评估翻译和语言方面的可能性，例如准确性和语法流畅性。