Effective communication is paramount for the inclusion of deaf individuals in
society. However, persistent communication barriers due to limited Sign
Language (SL) knowledge hinder their full participation. In this context, Sign
Language Recognition (SLR) systems have been developed to improve communication
between signing and non-signing individuals. In particular, there is the
problem of recognizing isolated signs (Isolated Sign Language Recognition,
ISLR) of great relevance in the development of vision-based SL search engines,
learning tools, and translation systems. This work proposes an ISLR approach
where body, hands, and facial landmarks are extracted throughout time and
encoded as 2-D images. These images are processed by a convolutional neural
network, which maps the visual-temporal information into a sign label.
Experimental results demonstrate that our method surpassed the state-of-the-art
in terms of performance metrics on two widely recognized datasets in Brazilian
Sign Language (LIBRAS), the primary focus of this study. In addition to being
more accurate, our method is more time-efficient and easier to train due to its
reliance on a simpler network architecture and solely RGB data as input.

通过提取身体、手部和面部关键点并将其编码为 2D 图像，本研究提出了一种孤立手语识别（ISLR）方法，通过卷积神经网络映射视觉和时间信息到手语标签，实验证明该方法在两个广为认可的巴西手语（LIBRAS）数据集上的性能指标超过了现有技术。此外，由于依赖于更简单的网络架构和仅使用 RGB 数据作为输入，本方法更准确、更高效且更易于训练。

通过骨架图像表示增强巴西手语识别

Enhancing Brazilian Sign Language Recognition through Skeleton Image  Representation

Sign language to text is a crucial technology that can break down
communication barriers for individuals with hearing difficulties. We replicate
and try to improve on a recently published study. We evaluate models using BLEU
and rBLEU metrics to ensure translation quality. During our ablation study, we
found that the model's performance is significantly influenced by optimizers,
activation functions, and label smoothing. Further research aims to refine
visual feature capturing, enhance decoder utilization, and integrate
pre-trained decoders for better translation outcomes. Our source code is
available to facilitate replication of our results and encourage future
research.

手语转文字是一项关键技术，可以消除听障者之间的交流障碍。我们在最近发表的研究上进行复制并进行改进，通过使用 BLEU 和 rBLEU 度量来评估模型的翻译质量。在我们的消融研究中，我们发现模型的性能受到优化器、激活函数和标签平滑的显著影响。进一步的研究旨在改进视觉特征捕捉、增强解码器的利用以及整合预训练解码器以获得更好的翻译结果。我们的源代码可供查看以便复制我们的结果并鼓励未来研究。

美国手语视频文本翻译

American Sign Language Video to Text Translation

We use insights from research on American Sign Language (ASL) phonology to
train models for isolated sign language recognition (ISLR), a step towards
automatic sign language understanding. Our key insight is to explicitly
recognize the role of phonology in sign production to achieve more accurate
ISLR than existing work which does not consider sign language phonology. We
train ISLR models that take in pose estimations of a signer producing a single
sign to predict not only the sign but additionally its phonological
characteristics, such as the handshape. These auxiliary predictions lead to a
nearly 9% absolute gain in sign recognition accuracy on the WLASL benchmark,
with consistent improvements in ISLR regardless of the underlying prediction
model architecture. This work has the potential to accelerate linguistic
research in the domain of signed languages and reduce communication barriers
between deaf and hearing people.

通过将语音学应用于手语产生过程中，训练出的模型预测手势及其语音学特征，从而实现对独立手语识别（ISLR）的模型训练，提高了在 WLASL 基准测试上的手势识别准确性达到了近 9% 的绝对增益，帮助缩小聋听者之间的沟通鸿沟。