Visual reconstruction algorithms are an interpretive tool that map brain
activity to pixels. Past reconstruction algorithms employed brute-force search
through a massive library to select candidate images that, when passed through
an encoding model, accurately predict brain activity. Here, we use conditional
generative diffusion models to extend and improve this search-based strategy.
We decode a semantic descriptor from human brain activity (7T fMRI) in voxels
across most of visual cortex, then use a diffusion model to sample a small
library of images conditioned on this descriptor. We pass each sample through
an encoding model, select the images that best predict brain activity, and then
use these images to seed another library. We show that this process converges
on high-quality reconstructions by refining low-level image details while
preserving semantic content across iterations. Interestingly, the
time-to-convergence differs systematically across visual cortex, suggesting a
succinct new way to measure the diversity of representations across visual
brain areas.

利用条件生成扩散模型对大脑活动中的语义描述符进行可视化重建，有效改善了搜索策略，同时保持了低级图像细节的精细度和语义信息的一致性，并且发现该过程的收敛时间与视觉皮层的区域差异有关。

通过引导性随机搜索从人脑活动中重建已见图像

Reconstructing seen images from human brain activity via guided  stochastic search

The number of categories for action recognition is growing rapidly and it has
become increasingly hard to label sufficient training data for learning
conventional models for all categories. Instead of collecting ever more data
and labelling them exhaustively for all categories, an attractive alternative
approach is zero-shot learning" (ZSL). To that end, in this study we construct
a mapping between visual features and a semantic descriptor of each action
category, allowing new categories to be recognised in the absence of any visual
training data. Existing ZSL studies focus primarily on still images, and
attribute-based semantic representations. In this work, we explore word-vectors
as the shared semantic space to embed videos and category labels for ZSL action
recognition. This is a more challenging problem than existing ZSL of still
images and/or attributes, because the mapping between video spacetime features
of actions and the semantic space is more complex and harder to learn for the
purpose of generalising over any cross-category domain shift. To solve this
generalisation problem in ZSL action recognition, we investigate a series of
synergistic strategies to improve upon the standard ZSL pipeline. Most of these
strategies are transductive in nature which means access to testing data in the
training phase.

本文探索了将视频和类别标签嵌入共享语义空间的方法，作为零样本学习（zero-shot learning）动作识别的一种解决方案，并提出了一系列策略来改善标准 ZSL 流程中的泛化问题，大多数策略在性质上是转导的，这意味着在训练阶段可以访问测试数据。

使用词向量嵌入的归纳零样本动作识别

Transductive Zero-Shot Action Recognition by Word-Vector Embedding

In this paper, we provide a new neural-network based perspective on
multi-task learning (MTL) and multi-domain learning (MDL). By introducing the
concept of a semantic descriptor, this framework unifies MDL and MTL as well as
encompassing various classic and recent MTL/MDL algorithms by interpreting them
as different ways of constructing semantic descriptors. Our interpretation
provides an alternative pipeline for zero-shot learning (ZSL), where a model
for a novel class can be constructed without training data. Moreover, it leads
to a new and practically relevant problem setting of zero-shot domain
adaptation (ZSDA), which is the analogous to ZSL but for novel domains: A model
for an unseen domain can be generated by its semantic descriptor. Experiments
across this range of problems demonstrate that our framework outperforms a
variety of alternatives.

本文提出了一种基于神经网络的新视角，并引入语义描述符的概念，从而将多任务学习和多域学习统一起来，并将各种经典和最新的多任务学习 / 多域学习算法解释为构建语义描述符的不同方式。通过本文的解释，可以提供一种零样本学习（ZSL）的替代方法，而且新的实践问题 —— 零样本域适应（ZSDA）也得以提出。实验表明，我们的框架在各种问题上都胜过了许多其他方法。