It has been observed that representations learned by distinct neural networks
conceal structural similarities when the models are trained under similar
inductive biases. From a geometric perspective, identifying the classes of
transformations and the related invariances that connect these representations
is fundamental to unlocking applications, such as merging, stitching, and
reusing different neural modules. However, estimating task-specific
transformations a priori can be challenging and expensive due to several
factors (e.g., weights initialization, training hyperparameters, or data
modality). To this end, we introduce a versatile method to directly incorporate
a set of invariances into the representations, constructing a product space of
invariant components on top of the latent representations without requiring
prior knowledge about the optimal invariance to infuse. We validate our
solution on classification and reconstruction tasks, observing consistent
latent similarity and downstream performance improvements in a zero-shot
stitching setting. The experimental analysis comprises three modalities
(vision, text, and graphs), twelve pretrained foundational models, eight
benchmarks, and several architectures trained from scratch.

直接将一组不变性直接融入表示中，构建一种不变分量的产品空间，旨在解锁合并、拼接和重用不同神经模块的应用，并观察到分类和重建任务中的一致潜在相似性和下游性能改进。

从砖头到桥梁：增强潜在空间通信的不变性乘积

From Bricks to Bridges: Product of Invariances to Enhance Latent Space  Communication

Neural networks embed the geometric structure of a data manifold lying in a
high-dimensional space into latent representations. Ideally, the distribution
of the data points in the latent space should depend only on the task, the
data, the loss, and other architecture-specific constraints. However, factors
such as the random weights initialization, training hyperparameters, or other
sources of randomness in the training phase may induce incoherent latent spaces
that hinder any form of reuse. Nevertheless, we empirically observe that, under
the same data and modeling choices, the angles between the encodings within
distinct latent spaces do not change. In this work, we propose the latent
similarity between each sample and a fixed set of anchors as an alternative
data representation, demonstrating that it can enforce the desired invariances
without any additional training. We show how neural architectures can leverage
these relative representations to guarantee, in practice, invariance to latent
isometries and rescalings, effectively enabling latent space communication:
from zero-shot model stitching to latent space comparison between diverse
settings. We extensively validate the generalization capability of our approach
on different datasets, spanning various modalities (images, text, graphs),
tasks (e.g., classification, reconstruction) and architectures (e.g., CNNs,
GCNs, transformers).

本文主要研究利用神经网络将数据流形的几何结构嵌入到潜在表征中，提出了根据潜在表示中样本与预设锚点之间的相对关系来实现所需不变性的方法，并在不同数据集、不同架构、不同任务下进行了验证。

相对表示使零样本隐空间通信成为可能

Relative representations enable zero-shot latent space communication

To study how mental object representations are related to behavior, we
estimated sparse, non-negative representations of objects using human
behavioral judgments on images representative of 1,854 object categories. These
representations predicted a latent similarity structure between objects, which
captured most of the explainable variance in human behavioral judgments.
Individual dimensions in the low-dimensional embedding were found to be highly
reproducible and interpretable as conveying degrees of taxonomic membership,
functionality, and perceptual attributes. We further demonstrated the
predictive power of the embeddings for explaining other forms of human
behavior, including categorization, typicality judgments, and feature ratings,
suggesting that the dimensions reflect human conceptual representations of
objects beyond the specific task.

通过人类行为判断对 1854 个物体类别的图像进行稀疏、非负表示，找出表达分类、功能和感知属性程度的低维嵌入的尽可能解释人类行为判断可变性的潜在相似性结构，从而展示了这些嵌入形态在阐释人类概念物体表示方面的预测能力。