This research explores using lightweight deep neural network architectures to
enable the humanoid robot Pepper to understand American Sign Language (ASL) and
facilitate non-verbal human-robot interaction. First, we introduce a
lightweight and efficient model for ASL understanding optimized for embedded
systems, ensuring rapid sign recognition while conserving computational
resources. Building upon this, we employ large language models (LLMs) for
intelligent robot interactions. Through intricate prompt engineering, we tailor
interactions to allow the Pepper Robot to generate natural Co-Speech Gesture
responses, laying the foundation for more organic and intuitive humanoid-robot
dialogues. Finally, we present an integrated software pipeline, embodying
advancements in a socially aware AI interaction model. Leveraging the Pepper
Robot's capabilities, we demonstrate the practicality and effectiveness of our
approach in real-world scenarios. The results highlight a profound potential
for enhancing human-robot interaction through non-verbal interactions, bridging
communication gaps, and making technology more accessible and understandable.

本研究旨在使用轻量级深度神经网络架构，使人型机器人 Pepper 能够理解美国手语（ASL），促进非语言人机交互。我们首先介绍了一个针对嵌入式系统进行了优化的轻量级高效 ASL 模型，可以在保留计算资源的同时快速识别手势。其次，我们运用大型语言模型（LLMs）进行智能机器人交互。通过精心设计的提示工程，我们能够实现 Pepper 机器人生成自然的共语手势回应，为更有机、直观的人机对话奠定基础。最后，我们提供了一个集成的软件管道，体现了社交意识型人工智能交互模型的最新进展。利用 Pepper 机器人的能力，我们在实际场景中验证了我们方法的实用性和有效性。结果突出了通过非语言交互增强人机交互的巨大潜力，缩小沟通差距，使技术更易接近和理解。