With the recent explosion of large language models (LLMs), such as Generative
Pretrained Transformers (GPT), the need to understand the ability of humans and
machines to comprehend semantic language meaning has entered a new phase. This
requires interdisciplinary research that bridges the fields of cognitive
science and natural language processing (NLP). This pilot study aims to provide
insights into individuals' neural states during a semantic relation
reading-comprehension task. We propose jointly analyzing LLMs, eye-gaze, and
electroencephalographic (EEG) data to study how the brain processes words with
varying degrees of relevance to a keyword during reading. We also use a feature
engineering approach to improve the fixation-related EEG data classification
while participants read words with high versus low relevance to the keyword.
The best validation accuracy in this word-level classification is over 60\%
across 12 subjects. Words of high relevance to the inference keyword had
significantly more eye fixations per word: 1.0584 compared to 0.6576 when
excluding no-fixation words, and 1.5126 compared to 1.4026 when including them.
This study represents the first attempt to classify brain states at a word
level using LLM knowledge. It provides valuable insights into human cognitive
abilities and the realm of Artificial General Intelligence (AGI), and offers
guidance for developing potential reading-assisted technologies.

通过联合分析大型语言模型、眼动数据和脑电图数据，本研究旨在研究人脑在阅读时对具有不同关联程度的关键词的词语的处理方式，并用特征工程方法改善相关脑电图数据的分类，从而为人类的认知能力和人工智能领域提供有价值的洞察，并为发展潜在的阅读辅助技术提供指导。

ChatGPT-BCI: 使用 GPT、EEG 和眼动生物标志的语义推理阅读理解的词级神经状态分类

ChatGPT-BCI: Word-Level Neural State Classification Using GPT, EEG, and  Eye-Tracking Biomarkers in Semantic Inference Reading Comprehension

We introduce adversarial neural networks for representation learning as a
novel approach to transfer learning in brain-computer interfaces (BCIs). The
proposed approach aims to learn subject-invariant representations by
simultaneously training a conditional variational autoencoder (cVAE) and an
adversarial network. We use shallow convolutional architectures to realize the
cVAE, and the learned encoder is transferred to extract subject-invariant
features from unseen BCI users' data for decoding. We demonstrate a
proof-of-concept of our approach based on analyses of electroencephalographic
(EEG) data recorded during a motor imagery BCI experiment.

本研究介绍了对抗神经网络在大脑 - 计算机界面（BCI）传递学习中作为表征学习的新方法。该方法旨在通过同时训练条件变分自动编码器（cVAE）和对抗网络来学习主体不变的表示，我们使用浅卷积结构实现 cVAE，并将学习的编码器转移到未见 BCI 用户的数据中。我们通过电子脑电图（EEG）数据对该方法进行了概念验证，其中记录了运动想象 BCI 实验时的数据。