Encoder models trained for the embedding of sentences or short documents have
proven useful for tasks such as semantic search and topic modeling. In this
paper, we present a version of the SwissBERT encoder model that we specifically
fine-tuned for this purpose. SwissBERT contains language adapters for the four
national languages of Switzerland -- German, French, Italian, and Romansh --
and has been pre-trained on a large number of news articles in those languages.
Using contrastive learning based on a subset of these articles, we trained a
fine-tuned version, which we call SentenceSwissBERT. Multilingual experiments
on document retrieval and text classification in a Switzerland-specific setting
show that SentenceSwissBERT surpasses the accuracy of the original SwissBERT
model and of a comparable baseline. The model is openly available for research
use.

对句子或短文档进行嵌入式编码的编码器模型，用于语义搜索和主题建模。本文介绍了一种专门为此目的微调的 SwissBERT 编码器模型版本，采用对瑞士四种国家语言（德语、法语、意大利语和罗曼什语）进行了预训练，使用对应文章子集的对比学习进行微调，实验证明 SentenceSwissBERT 在瑞士特定环境下的文档检索和文本分类任务中表现对原始 SwissBERT 模型和可比基准模型的准确性。该模型可供研究使用。

对 SwissBERT 编码模型进行微调，用于嵌入句子和文档

Fine-tuning the SwissBERT Encoder Model for Embedding Sentences and  Documents

We present SwissBERT, a masked language model created specifically for
processing Switzerland-related text. SwissBERT is a pre-trained model that we
adapted to news articles written in the national languages of Switzerland --
German, French, Italian, and Romansh. We evaluate SwissBERT on natural language
understanding tasks related to Switzerland and find that it tends to outperform
previous models on these tasks, especially when processing contemporary news
and/or Romansh Grischun. Since SwissBERT uses language adapters, it may be
extended to Swiss German dialects in future work. The model and our open-source
code are publicly released at this https URL.

SwissBERT 是一个针对处理与瑞士有关的文本而创建的掩模语言模型，它是一个预训练模型，适用于瑞士的德语、法语、意大利语和罗曼什语的新闻文章，该模型使用语言适配器，在处理当代新闻和 / 或罗曼什・格里施人时，往往可以比以前的模型更好地完成与瑞士有关的自然语言理解任务，模型和开源代码在公共网址上发布。