In this paper, we introduce a massively multilingual speech corpora with
fine-grained phonemic transcriptions, encompassing more than 115 languages from
diverse language families. Based on this multilingual dataset, we propose
CLAP-IPA, a multilingual phoneme-speech contrastive embedding model capable of
open-vocabulary matching between speech signals and phonemically transcribed
keywords or arbitrary phrases. The proposed model has been tested on two
fieldwork speech corpora in 97 unseen languages, exhibiting strong
generalizability across languages. Comparison with a text-based model shows
that using phonemes as modeling units enables much better crosslinguistic
generalization than orthographic texts.

这篇论文介绍了一种包含 115 多种不同语系的细粒度音素转录的大规模多语言语音语料库。基于这个多语言数据集，我们提出了 CLAP-IPA，这是一种能够在语音信号和音素转录的关键词或任意短语之间进行开放词汇匹配的多语言音素语音对比嵌入模型。所提出的模型已在 97 种未见过的语言中的两个实地语音语料库上进行了测试，展示了跨语言的强大泛化能力。与基于文本的模型相比，使用音素作为建模单位比正字文本具有更好的跨语言泛化能力。