The process of reconstructing experiences from human brain activity offers a
unique lens into how the brain interprets and represents the world. In this
paper, we introduce a method for reconstructing music from brain activity,
captured using functional magnetic resonance imaging (fMRI). Our approach uses
either music retrieval or the MusicLM music generation model conditioned on
embeddings derived from fMRI data. The generated music resembles the musical
stimuli that human subjects experienced, with respect to semantic properties
like genre, instrumentation, and mood. We investigate the relationship between
different components of MusicLM and brain activity through a voxel-wise
encoding modeling analysis. Furthermore, we discuss which brain regions
represent information derived from purely textual descriptions of music
stimuli. We provide supplementary material including examples of the
reconstructed music at this https URL

通过功能性磁共振成像（fMRI）捕获的脑活动，我们介绍了一种从脑活动重构音乐的方法，该方法使用音乐检索或 MusicLM 音乐生成模型，根据 fMRI 数据的嵌入来生成类似于人类主体经历的音乐，包括语义特征如流派、乐器和情绪。我们通过体素编码模型分析，探讨了 MusicLM 的不同组件与脑活动之间的关系，并讨论了哪些脑区表示来自纯文本音乐描述的信息。我们提供了补充材料，包括重构音乐的示例。