Sign Language Recognition (SLR) is a fast-growing field that aims to fill the
communication gaps between the hearing-impaired and people without hearing
loss. Existing solutions for Persian Sign Language (PSL) are limited to
word-level interpretations, underscoring the need for more advanced and
comprehensive solutions. Moreover, previous work on other languages mainly
focuses on manipulating the neural network architectures or hardware
configurations instead of benefiting from the aggregated results of multiple
models. In this paper, we introduce PenSLR, a glove-based sign language system
consisting of an Inertial Measurement Unit (IMU) and five flexible sensors
powered by a deep learning framework capable of predicting variable-length
sequences. We achieve this in an end-to-end manner by leveraging the
Connectionist Temporal Classification (CTC) loss function, eliminating the need
for segmentation of input signals. To further enhance its capabilities, we
propose a novel ensembling technique by leveraging a multiple sequence
alignment algorithm known as Star Alignment. Furthermore, we introduce a new
PSL dataset, including 16 PSL signs with more than 3000 time-series samples in
total. We utilize this dataset to evaluate the performance of our system based
on four word-level and sentence-level metrics. Our evaluations show that PenSLR
achieves a remarkable word accuracy of 94.58% and 96.70% in subject-independent
and subject-dependent setups, respectively. These achievements are attributable
to our ensembling algorithm, which not only boosts the word-level performance
by 0.51% and 1.32% in the respective scenarios but also yields significant
enhancements of 1.46% and 4.00%, respectively, in sentence-level accuracy.

本文介绍了 PenSLR，一种基于手套的手语识别系统，采用惯性测量单元（IMU）和五个柔性传感器，结合深度学习框架以及利用连接主义时序分类（CTC）损失函数，通过多序列比对算法提供的新方法进行集成，优化其在波斯手语上的性能表现，实验结果显示 PenSLR 在单词准确率和句子准确率方面都取得了令人满意的成果。