Recent strides in multimodal model development have ignited a paradigm shift
in the realm of text-to-image generation. Among these advancements, CLIP stands
out as a remarkable achievement which is a sophisticated autoencoder adept at
encoding both textual and visual information within a unified latent space.
This paper delves into a comparative analysis between CLIP and its recent
counterpart, CLOOB. To unravel the intricate distinctions within the embedding
spaces crafted by these models, we employ topological data analysis. Our
approach encompasses a comprehensive examination of the modality gap drivers,
the clustering structures existing across both high and low dimensions, and the
pivotal role that dimension collapse plays in shaping their respective
embedding spaces. Empirical experiments substantiate the implications of our
analyses on downstream performance across various contextual scenarios. Through
this investigation, we aim to shed light on the nuanced intricacies that
underlie the comparative efficacy of CLIP and CLOOB, offering insights into
their respective strengths and weaknesses, and providing a foundation for
further refinement and advancement in multimodal model research.

通过对 CLIP 和 CLOOB 模型嵌入空间的拓扑数据分析，研究探索了多模态模型发展及其下游性能的关联，揭示了嵌入空间中的细微差异，为进一步改进和推进多模态模型研究提供了基础。