In this paper, we introduce a groundbreaking end-to-end (E2E) framework for
decoding invasive brain signals, marking a significant advancement in the field
of speech neuroprosthesis. Our methodology leverages the comprehensive
reasoning abilities of large language models (LLMs) to facilitate direct
decoding. By fully integrating LLMs, we achieve results comparable to the
state-of-the-art cascade models. Our findings underscore the immense potential
of E2E frameworks in speech neuroprosthesis, particularly as the technology
behind brain-computer interfaces (BCIs) and the availability of relevant
datasets continue to evolve. This work not only showcases the efficacy of
combining LLMs with E2E decoding for enhancing speech neuroprosthesis but also
sets a new direction for future research in BCI applications, underscoring the
impact of LLMs in decoding complex neural signals for communication
restoration. Code will be made available at
this https URL

介绍了一种突破性的端到端（E2E）框架，用于解码侵入性脑信号，取得了语音神经假肢领域的重大进展。通过充分整合大型语言模型，实现与最先进的级联模型相当的结果，突出了 E2E 框架在语音神经假肢中的巨大潜力，尤其在脑机接口技术和相关数据集的发展进化方面。该项工作不仅展示了将大型语言模型与 E2E 解码相结合以增强语音神经假肢的有效性，还为脑机接口应用的未来研究指明了新的方向，突显了在解码复杂神经信号以实现恢复交流方面，大型语言模型的影响力。