Brain signal visualization has emerged as an active research area, serving as
a critical interface between the human visual system and computer vision
models. Although diffusion models have shown promise in analyzing functional
magnetic resonance imaging (fMRI) data, including reconstructing high-quality
images consistent with original visual stimuli, their accuracy in extracting
semantic and silhouette information from brain signals remains limited. In this
regard, we propose a novel approach, referred to as Controllable Mind Visual
Diffusion Model (CMVDM). CMVDM extracts semantic and silhouette information
from fMRI data using attribute alignment and assistant networks. Additionally,
a residual block is incorporated to capture information beyond semantic and
silhouette features. We then leverage a control model to fully exploit the
extracted information for image synthesis, resulting in generated images that
closely resemble the visual stimuli in terms of semantics and silhouette.
Through extensive experimentation, we demonstrate that CMVDM outperforms
existing state-of-the-art methods both qualitatively and quantitatively.

本文提出了一种名为 Controllable Mind Visual Diffusion Model (CMVDM) 的新方法，利用属性对齐和辅助网络从 fMRI 数据中提取语义和轮廓信息，并利用控制模型对提取的信息进行图像合成，与其他现有方法相比，CMVDM 在定性和定量上都具有更好的性能。