Recent approaches to Sign Language Production (SLP) have adopted spoken
language Neural Machine Translation (NMT) architectures, applied without
sign-specific modifications. In addition, these works represent sign language
as a sequence of skeleton pose vectors, projected to an abstract representation
with no inherent skeletal structure. In this paper, we represent sign language
sequences as a skeletal graph structure, with joints as nodes and both spatial
and temporal connections as edges. To operate on this graphical structure, we
propose Skeletal Graph Self-Attention (SGSA), a novel graphical attention layer
that embeds a skeleton inductive bias into the SLP model. Retaining the
skeletal feature representation throughout, we directly apply a spatio-temporal
adjacency matrix into the self-attention formulation. This provides structure
and context to each skeletal joint that is not possible when using a
non-graphical abstract representation, enabling fluid and expressive sign
language production. We evaluate our Skeletal Graph Self-Attention architecture
on the challenging RWTH-PHOENIX-Weather-2014T(PHOENIX14T) dataset, achieving
state-of-the-art back translation performance with an 8% and 7% improvement
over competing methods for the dev and test sets.

本文提出了一种将手语序列表示为包含节点和边的骨骼图结构，并运用 Skeletal Graph Self-Attention（SGSA）方法实现手语的流畅和表达，该方法在 RWTH-PHOENIX-Weather-2014T 数据集上展现出比其它竞争方法更强的后向翻译表现。

骨架图自注意力：将骨骼归纳偏差嵌入手语生成中

Skeletal Graph Self-Attention: Embedding a Skeleton Inductive Bias into  Sign Language Production

The goal of automatic Sign Language Production (SLP) is to translate spoken
language to a continuous stream of sign language video at a level comparable to
a human translator. If this was achievable, then it would revolutionise Deaf
hearing communications. Previous work on predominantly isolated SLP has shown
the need for architectures that are better suited to the continuous domain of
full sign sequences.
In this paper, we propose Progressive Transformers, a novel architecture that
can translate from discrete spoken language sentences to continuous 3D skeleton
pose outputs representing sign language. We present two model configurations,
an end-to-end network that produces sign direct from text and a stacked network
that utilises a gloss intermediary.
Our transformer network architecture introduces a counter that enables
continuous sequence generation at training and inference. We also provide
several data augmentation processes to overcome the problem of drift and
improve the performance of SLP models. We propose a back translation evaluation
mechanism for SLP, presenting benchmark quantitative results on the challenging
RWTH-PHOENIX-Weather-2014T(PHOENIX14T) dataset and setting baselines for future
research.

该论文提出了一种名为 Progressive Transformers 的新型架构，旨在将离散文本语言句子翻译成表示手语的连续 3D 骨架姿势输出，从而实现自动手语翻译，作者在文中还提出了几种数据增强处理方式以克服漂移问题并提高手语翻译性能，在 RWTH-PHOENIX-Weather-2014T 数据集上取得了令人瞩目的基础性成果，为未来的研究提供了基线。