Visually Impaired Assistance (VIA) aims to automatically help visually
impaired (VI) handle daily activities. The advancement of VIA primarily depends
on developments in Computer Vision (CV) and Natural Language Processing (NLP),
both of which exhibit cutting-edge paradigms with large models (LMs).
Furthermore, LMs have shown exceptional multimodal abilities to tackle
challenging physically-grounded tasks such as embodied robots. To investigate
the potential and limitations of state-of-the-art (SOTA) LMs' capabilities in
VIA applications, we present an extensive study for the task of VIA with LMs
(\textbf{VIALM}). In this task, given an \textit{image} illustrating the
physical environments and a \textit{linguistic request} from a VI user, VIALM
aims to output step-by-step \textit{guidance} to assist the VI user in
fulfilling the request grounded in the environment. The study consists of a
survey reviewing recent LM research and benchmark experiments examining
selected LMs' capabilities in VIA. The results indicate that while LMs can
augment VIA, their output cannot be well \textit{grounded} in reality (i.e.,
25.7\% GPT-4's responses) and lacks \textit{fine-grained} guidance (i.e.,
32.1\% GPT-4's responses).

自动化帮助视障人士处理日常活动的目标是通过计算机视觉和自然语言处理的发展实现的，使用大型模型，通过大规模研究调查了大型模型在视障辅助方面的潜力和限制，结果显示，虽然大型模型可以增强视障辅助功能，但其输出无法很好地与现实接轨并且缺乏细致的指导。