Semantic Retrieval (SR) has become an indispensable part of the FAQ system in
the task-oriented question-answering (QA) dialogue scenario. The demands for a
cross-lingual smart-customer-service system for an e-commerce platform or some
particular business conditions have been increasing recently. Most previous
studies exploit cross-lingual pre-trained models (PTMs) for multi-lingual
knowledge retrieval directly, while some others also leverage the continual
pre-training before fine-tuning PTMs on the downstream tasks. However, no
matter which schema is used, the previous work ignores to inform PTMs of some
features of the downstream task, i.e. train their PTMs without providing any
signals related to SR. To this end, in this work, we propose an Alternative
Cross-lingual PTM for SR via code-switching. We are the first to utilize the
code-switching approach for cross-lingual SR. Besides, we introduce the novel
code-switched continual pre-training instead of directly using the PTMs on the
SR tasks. The experimental results show that our proposed approach consistently
outperforms the previous SOTA methods on SR and semantic textual similarity
(STS) tasks with three business corpora and four open datasets in 20+
languages.

本研究提出了一种基于代码切换的备选跨语言 PTM 方法用于语义检索，通过引入代码切换的持续预训练，相对于直接使用 PTM 在语义检索任务上的方法，我们的方法在二十多种语言的三个商业语料库和四个开放数据集上连续超过了之前的 SOTA 方法。