The visually impaired population, especially the severely visually impaired, is currently large in scale, and daily activities pose significant challenges for them. Although many studies use large language and vision-language models to assist the blind, most focus on static content and fail to meet real-time perception needs in dynamic and complex environments, such as daily activities. To provide them with more effective intelligent assistance, it is imperative to incorporate advanced visual understanding technologies. Although real-time vision and speech interaction VideoLLMs demonstrate strong real-time visual understanding, no prior work has systematically evaluated their effectiveness in assisting visually impaired individuals. In this work, we conduct the first such evaluation. First, we construct a benchmark dataset (VisAssistDaily), covering three categories of assistive tasks for visually impaired individuals: Basic Skills, Home Life Tasks, and Social Life Tasks. The results show that GPT-4o achieves the highest task success rate. Next, we conduct a user study to evaluate the models in both closed-world and open-world scenarios, further exploring the practical challenges of applying VideoLLMs in assistive contexts. One key issue we identify is the difficulty current models face in perceiving potential hazards in dynamic environments. To address this, we build an environment-awareness dataset named SafeVid and introduce a polling mechanism that enables the model to proactively detect environmental risks. We hope this work provides valuable insights and inspiration for future research in this field.

本研究解决了视觉障碍者在动态复杂环境中日常活动中的实时感知需求尚未得到满足的难题。通过构建一个基准数据集（VisAssistDaily），我们评估了实时视频语言模型在辅助视觉障碍者的有效性，发现GPT-4o在任务成功率上表现最佳。我们还提出了环境意识数据集SafeVid及轮询机制，以更好地检测动态环境中的潜在危险，为未来研究提供了有价值的见解与灵感。

“我能看到永恒！”：评估实时视频语言模型以协助视觉障碍者