Generating human language through non-invasive brain-computer interfaces
(BCIs) has the potential to unlock many applications, such as serving disabled
patients and improving communication. Currently, however, generating language
via BCIs has been previously successful only within a classification setup for
selecting pre-generated sentence continuation candidates with the most likely
cortical semantic representation. Inspired by recent research that revealed
associations between the brain and the large computational language models, we
propose a generative language BCI that utilizes the capacity of a large
language model (LLM) jointly with a semantic brain decoder to directly generate
language from functional magnetic resonance imaging (fMRI) input. The proposed
model can generate coherent language sequences aligned with the semantic
content of visual or auditory language stimuli perceived, without prior
knowledge of any pre-generated candidates. We compare the language generated
from the presented model with a random control, pre-generated language
selection approach, and a standard LLM, which generates common coherent text
solely based on the next word likelihood according to statistical language
training data. The proposed model is found to generate language that is more
aligned with semantic stimulus in response to which brain input is sampled. Our
findings demonstrate the potential and feasibility of employing BCIs in direct
language generation.

通过使用大规模语言模型（LLM）与语义脑解码器相结合，直接从功能性磁共振成像（fMRI）输入中生成与视觉或听觉语言刺激内容紧密相关的连贯语言序列，这证明了利用脑机接口（BCIs）直接生成语言的潜力和可行性。