Communicative efficiency is a prominent theory in linguistics and cognitive
science. While numerous studies have shown how the pressure to save energy is
reflected in the form of spoken languages, few have explored this phenomenon in
signed languages. In this paper, we show how handshapes in American Sign
Language (ASL) reflect these efficiency pressures and we present new evidence
of communicative efficiency in the visual-gestural modality.
We focus on handshapes that are used in both native ASL signs and signs
borrowed from English to compare efficiency pressures from both ASL and
English. First, we design new methodologies to quantify the articulatory effort
required to produce handshapes as well as the perceptual effort needed to
recognize them. Then, we compare correlations between communicative effort and
usage statistics in ASL and English. Our findings reveal that frequent ASL
handshapes are easier to produce and that pressures for communicative
efficiency mostly come from ASL usage, not from English lexical borrowing.

通过比较美国手语（ASL）和英语中使用的手形，该研究表明频繁使用的手形更容易产生，且沟通效率的压力主要来自 ASL 的使用，而非英语的借用。

美国手语手势反映了沟通效率的压力

American Sign Language Handshapes Reflect Pressures for Communicative  Efficiency

This research explores using lightweight deep neural network architectures to
enable the humanoid robot Pepper to understand American Sign Language (ASL) and
facilitate non-verbal human-robot interaction. First, we introduce a
lightweight and efficient model for ASL understanding optimized for embedded
systems, ensuring rapid sign recognition while conserving computational
resources. Building upon this, we employ large language models (LLMs) for
intelligent robot interactions. Through intricate prompt engineering, we tailor
interactions to allow the Pepper Robot to generate natural Co-Speech Gesture
responses, laying the foundation for more organic and intuitive humanoid-robot
dialogues. Finally, we present an integrated software pipeline, embodying
advancements in a socially aware AI interaction model. Leveraging the Pepper
Robot's capabilities, we demonstrate the practicality and effectiveness of our
approach in real-world scenarios. The results highlight a profound potential
for enhancing human-robot interaction through non-verbal interactions, bridging
communication gaps, and making technology more accessible and understandable.

本研究旨在使用轻量级深度神经网络架构，使人型机器人 Pepper 能够理解美国手语（ASL），促进非语言人机交互。我们首先介绍了一个针对嵌入式系统进行了优化的轻量级高效 ASL 模型，可以在保留计算资源的同时快速识别手势。其次，我们运用大型语言模型（LLMs）进行智能机器人交互。通过精心设计的提示工程，我们能够实现 Pepper 机器人生成自然的共语手势回应，为更有机、直观的人机对话奠定基础。最后，我们提供了一个集成的软件管道，体现了社交意识型人工智能交互模型的最新进展。利用 Pepper 机器人的能力，我们在实际场景中验证了我们方法的实用性和有效性。结果突出了通过非语言交互增强人机交互的巨大潜力，缩小沟通差距，使技术更易接近和理解。

一种基于 Pepper、Lightweight-Transformer 和 LLM 的手语识别系统

A Sign Language Recognition System with Pepper, Lightweight-Transformer,  and LLM

Machine learning for sign languages is bottlenecked by data. In this paper,
we present YouTube-ASL, a large-scale, open-domain corpus of American Sign
Language (ASL) videos and accompanying English captions drawn from YouTube.
With ~1000 hours of videos and >2500 unique signers, YouTube-ASL is ~3x as
large and has ~10x as many unique signers as the largest prior ASL dataset. We
train baseline models for ASL to English translation on YouTube-ASL and
evaluate them on How2Sign, where we achieve a new finetuned state of the art of
12.39 BLEU and, for the first time, report zero-shot results.

本论文通过 Youtube-ASL，一个大规模的开放领域美国手语（ASL）视频语料库的介绍及研究，证明了手语机器学习的瓶颈在于数据问题，并使用 How2Sign 进行了模型基线培训及评估，并获得了新的微调技术的艺术的最高状态和首次报道的零 - shot 结果。