An autoassociative memory model is a function that, given a set of data
points, takes as input an arbitrary vector and outputs the most similar data
point from the memorized set. However, popular memory models fail to retrieve
images even when the corruption is mild and easy to detect for a human
evaluator. This is because similarities are evaluated in the raw pixel space,
which does not contain any semantic information about the images. This problem
can be easily solved by computing \emph{similarities} in an embedding space
instead of the pixel space. We show that an effective way of computing such
embeddings is via a network pretrained with a contrastive loss. As the
dimension of embedding spaces is often significantly smaller than the pixel
space, we also have a faster computation of similarity scores. We test this
method on complex datasets such as CIFAR10 and STL10. An additional drawback of
current models is the need of storing the whole dataset in the pixel space,
which is often extremely large. We relax this condition and propose a class of
memory models that only stores low-dimensional semantic embeddings, and uses
them to retrieve similar, but not identical, memories. We demonstrate a proof
of concept of this method on a simple task on the MNIST dataset.

通过使用嵌入空间计算相似度，而不是像素空间，利用预训练的对比损失网络来计算嵌入，我们可以在较小的嵌入空间中更快地计算相似度分数，并且不需要存储整个数据集的像素空间。我们还提出了一种仅存储低维语义嵌入的记忆模型类别，并使用它们来检索类似但不完全相同的记忆。在 MNIST 数据集上的简单任务上，我们对这种方法进行了概念验证。

特征空间中的关联记忆

Associative Memories in the Feature Space

Multimodal variational autoencoders (VAEs) have shown promise as efficient
generative models for weakly-supervised data. Yet, despite their advantage of
weak supervision, they exhibit a gap in generative quality compared to unimodal
VAEs, which are completely unsupervised. In an attempt to explain this gap, we
uncover a fundamental limitation that applies to a large family of
mixture-based multimodal VAEs. We prove that the sub-sampling of modalities
enforces an undesirable upper bound on the multimodal ELBO and thereby limits
the generative quality of the respective models. Empirically, we showcase the
generative quality gap on both synthetic and real data and present the
tradeoffs between different variants of multimodal VAEs. We find that none of
the existing approaches fulfills all desired criteria of an effective
multimodal generative model when applied on more complex datasets than those
used in previous benchmarks. In summary, we identify, formalize, and validate
fundamental limitations of VAE-based approaches for modeling weakly-supervised
data and discuss implications for real-world applications.

研究发现，多模态变分自编码器存在上界限制，导致其生成质量不如完全无监督的单模态自编码器，且当前的算法无法满足所有复杂数据集上的期望表现，限制了其在弱监督数据建模应用中的实际效果。