A persistent challenge in sign language video processing, including the task
of sign language to written language translation, is how we learn
representations of sign language in an effective and efficient way that can
preserve the important attributes of these languages, while remaining invariant
to irrelevant visual differences. Informed by the nature and linguistics of
signed languages, our proposed method focuses on just the most relevant parts
in a signing video: the face, hands and body posture of the signer. However,
instead of using pose estimation coordinates from off-the-shelf pose tracking
models, which have inconsistent performance for hands and faces, we propose to
learn the complex handshapes and rich facial expressions of sign languages in a
self-supervised fashion. Our approach is based on learning from individual
frames (rather than video sequences) and is therefore much more efficient than
prior work on sign language pre-training. Compared to a recent model that
established a new state of the art in sign language translation on the How2Sign
dataset, our approach yields similar translation performance, using less than
3\% of the compute.

基于影像及语言学特性，本研究提出了一种有效且高效的学习手语表示的方法，专注于手势视频中的脸部、手部和身体姿态，并通过自监督学习的方式学习手语中复杂的手形和丰富的面部表情，实现对手语翻译的新突破。

SignMusketeers：大规模手语翻译的高效多流方法

SignMusketeers: An Efficient Multi-Stream Approach for Sign Language  Translation at Scale

Communicative efficiency is a prominent theory in linguistics and cognitive
science. While numerous studies have shown how the pressure to save energy is
reflected in the form of spoken languages, few have explored this phenomenon in
signed languages. In this paper, we show how handshapes in American Sign
Language (ASL) reflect these efficiency pressures and we present new evidence
of communicative efficiency in the visual-gestural modality.
We focus on handshapes that are used in both native ASL signs and signs
borrowed from English to compare efficiency pressures from both ASL and
English. First, we design new methodologies to quantify the articulatory effort
required to produce handshapes as well as the perceptual effort needed to
recognize them. Then, we compare correlations between communicative effort and
usage statistics in ASL and English. Our findings reveal that frequent ASL
handshapes are easier to produce and that pressures for communicative
efficiency mostly come from ASL usage, not from English lexical borrowing.

通过比较美国手语（ASL）和英语中使用的手形，该研究表明频繁使用的手形更容易产生，且沟通效率的压力主要来自 ASL 的使用，而非英语的借用。