Translating spoken languages into Sign languages is necessary for open
communication between the hearing and hearing-impaired communities. To achieve
this goal, we propose the first method for animating a text written in
HamNoSys, a lexical Sign language notation, into signed pose sequences. As
HamNoSys is universal, our proposed method offers a generic solution invariant
to the target Sign language. Our method gradually generates pose predictions
using transformer encoders that create meaningful representations of the text
and poses while considering their spatial and temporal information. We use weak
supervision for the training process and show that our method succeeds in
learning from partial and inaccurate data. Additionally, we offer a new
distance measurement for pose sequences, normalized Dynamic Time Warping
(nDTW), based on DTW over normalized keypoints trajectories, and validate its
correctness using AUTSL, a large-scale Sign language dataset. We show that it
measures the distance between pose sequences more accurately than existing
measurements and use it to assess the quality of our generated pose sequences.
Code for the data pre-processing, the model, and the distance measurement is
publicly released for future research.

提出一种将 HamNoSys 词汇符号转换为手语姿势动画的方法，利用 Transformer 编码器逐渐生成动作预测并使用距离测量方法评估质量，为手语和听障人士之间的翻译提供一种通用解决方案。

Ham2Pose：将手语符号动画化为姿势序列

Ham2Pose: Animating Sign Language Notation into Pose Sequences

This paper presents our recent developments in the automatic processing of
sign language corpora using the Hamburg Sign Language Annotation System
(HamNoSys). We designed an automated tool to convert HamNoSys annotations into
numerical labels for defined initial features of body and hand positions. Our
proposed numerical multilabels greatly simplify annotations' structure without
significant loss of gloss meaning. These numerical multilabels can potentially
be used to feed the machine learning models, which would accelerate the
development of vision-based sign language recognition. In addition, this tool
can assist experts in the annotation process and help identify semantic errors.
The code and sample annotations are publicly available at
https://github.com/hearai/parse-hamnosys.

本文介绍了使用 Hamburg 手语注释系统（HamNoSys）对手语语料库进行自动处理的最新进展。我们设计了自动化工具，将 HamNoSys 注释转换为身体和手部位置的定义初始特征的数字标签。我们提出的数字多标签可以潜在地用于馈送机器学习模型，这将加速视觉手语识别的发展。此外，该工具可以协助专家进行注释过程，并帮助识别语义错误。