This paper presents M3L-Contrast -- a novel multimodal multilingual (M3L)
neural topic model for comparable data that maps texts from multiple languages
and images into a shared topic space. Our model is trained jointly on texts and
images and takes advantage of pretrained document and image embeddings to
abstract the complexities between different languages and modalities. As a
multilingual topic model, it produces aligned language-specific topics and as
multimodal model, it infers textual representations of semantic concepts in
images. We demonstrate that our model is competitive with a zero-shot topic
model in predicting topic distributions for comparable multilingual data and
significantly outperforms a zero-shot model in predicting topic distributions
for comparable texts and images. We also show that our model performs almost as
well on unaligned embeddings as it does on aligned embeddings.

本文介绍了一种新颖的多模态多语言（M3L）神经话题模型 M3L-Contrast，用于可比较数据，将多种语言和图像映射到共享的话题空间中，该模型利用预训练的文本和图像嵌入，抽象不同语言和模态之间的复杂性，并产生对齐的语言特定话题。同时，该模型还可以推断图像中语义概念的文本表示。我们证明了我们的模型在预测可比较的多语言数据的话题分布方面与零 - shot 模型相当，且在预测可比较的文本和图像的话题分布方面比零 - shot 模型表现更好。同时，我们还展示了我们的模型在未对齐嵌入上的表现几乎与在对齐嵌入上的表现一样好。