Two recent developments have accelerated progress in image reconstruction
from human brain activity: large datasets that offer samples of brain activity
in response to many thousands of natural scenes, and the open-sourcing of
powerful stochastic image-generators that accept both low- and high-level
guidance. Most work in this space has focused on obtaining point estimates of
the target image, with the ultimate goal of approximating literal pixel-wise
reconstructions of target images from the brain activity patterns they evoke.
This emphasis belies the fact that there is always a family of images that are
equally compatible with any evoked brain activity pattern, and the fact that
many image-generators are inherently stochastic and do not by themselves offer
a method for selecting the single best reconstruction from among the samples
they generate. We introduce a novel reconstruction procedure (Second Sight)
that iteratively refines an image distribution to explicitly maximize the
alignment between the predictions of a voxel-wise encoding model and the brain
activity patterns evoked by any target image. We show that our process
converges on a distribution of high-quality reconstructions by refining both
semantic content and low-level image details across iterations. Images sampled
from these converged image distributions are competitive with state-of-the-art
reconstruction algorithms. Interestingly, the time-to-convergence varies
systematically across visual cortex, with earlier visual areas generally taking
longer and converging on narrower image distributions, relative to higher-level
brain areas. Second Sight thus offers a succinct and novel method for exploring
the diversity of representations across visual brain areas.

提出了一种名为 Second Sight 的新型重建过程，该过程通过迭代方式显式最大化挑战任何目标图像所引起的脑活动模式的体素编码模型的预测与脑活动模式之间的对齐，从而在语义内容和低级图像细节方面不断改进损伤图像，解决了多种输入图像问题。