Do different generative image models secretly learn similar underlying representations? We investigate this by measuring the latent space similarity of four different models: VAEs, GANs, Normalizing Flows (NFs), and Diffusion Models (DMs). Our methodology involves training linear maps between frozen latent spaces to "stitch" arbitrary pairs of encoders and decoders and measuring output-based and probe-based metrics on the resulting "stitched'' models. Our main findings are that linear maps between latent spaces of performant models preserve most visual information even when latent sizes differ; for CelebA models, gender is the most similarly represented probe-able attribute. Finally we show on an NF that latent space representations converge early in training.

不同生成图像模型是否暗含相似的潜在表示？我们通过测量四种不同模型（VAEs、GANs、NFs和DMs）的潜在空间相似性来进行调查。我们的方法是使用冻结的潜在空间之间的线性映射，将任意配对的编码器和解码器连接起来，并测量生成的“连接”模型的基于输出和基于探针的度量。我们的主要发现是，性能良好的模型之间的潜在空间线性映射保留了大部分视觉信息，即使潜在空间大小不同；对于CelebA模型来说，性别是最相似的属性。最后，我们展示了NF上的实验结果表明，潜在空间表示在训练早期就会收敛。

所有道路通往罗马？探索生成图像模型的潜在空间之间的表征相似性