Visual reconstruction algorithms are an interpretive tool that map brain
activity to pixels. Past reconstruction algorithms employed brute-force search
through a massive library to select candidate images that, when passed through
an encoding model, accurately predict brain activity. Here, we use conditional
generative diffusion models to extend and improve this search-based strategy.
We decode a semantic descriptor from human brain activity (7T fMRI) in voxels
across most of visual cortex, then use a diffusion model to sample a small
library of images conditioned on this descriptor. We pass each sample through
an encoding model, select the images that best predict brain activity, and then
use these images to seed another library. We show that this process converges
on high-quality reconstructions by refining low-level image details while
preserving semantic content across iterations. Interestingly, the
time-to-convergence differs systematically across visual cortex, suggesting a
succinct new way to measure the diversity of representations across visual
brain areas.

利用条件生成扩散模型对大脑活动中的语义描述符进行可视化重建，有效改善了搜索策略，同时保持了低级图像细节的精细度和语义信息的一致性，并且发现该过程的收敛时间与视觉皮层的区域差异有关。