We introduce a new Slovak masked language model called SlovakBERT. This is to
our best knowledge the first paper discussing Slovak transformers-based
language models. We evaluate our model on several NLP tasks and achieve
state-of-the-art results. This evaluation is likewise the first attempt to
establish a benchmark for Slovak language models. We publish the masked
language model, as well as the fine-tuned models for part-of-speech tagging,
sentiment analysis and semantic textual similarity.

介绍了一种名为 SlovakBERT 的新型斯洛伐克遮蔽语言模型，是第一篇讨论基于 transformers 的斯洛伐克语言模型的论文。评估发现该模型在多项自然语言处理任务中取得了最优结果，并建立了斯洛伐克语言模型的基准。发布了遮蔽语言模型以及针对词性标注、情感分析和语义文本相似性的微调模型。

SlovakBERT：斯洛伐克遮罩语言模型

SlovakBERT: Slovak Masked Language Model

Amid the pandemic COVID-19, the world is facing unprecedented infodemic with
the proliferation of both fake and real information. Considering the
problematic consequences that the COVID-19 fake-news have brought, the
scientific community has put effort to tackle it. To contribute to this fight
against the infodemic, we aim to achieve a robust model for the COVID-19
fake-news detection task proposed at CONSTRAINT 2021 (FakeNews-19) by taking
two separate approaches: 1) fine-tuning transformers based language models with
robust loss functions and 2) removing harmful training instances through
influence calculation. We further evaluate the robustness of our models by
evaluating on different COVID-19 misinformation test set (Tweets-19) to
understand model generalization ability. With the first approach, we achieve
98.13% for weighted F1 score (W-F1) for the shared task, whereas 38.18% W-F1 on
the Tweets-19 highest. On the contrary, by performing influence data cleansing,
our model with 99% cleansing percentage can achieve 54.33% W-F1 score on
Tweets-19 with a trade-off. By evaluating our models on two COVID-19 fake-news
test sets, we suggest the importance of model generalization ability in this
task to step forward to tackle the COVID-19 fake-news problem in online social
media platforms.

为了抗击 COVID-19 疫情期间社交媒体上的虚假信息，本文提出两种方法，一是 fine-tuning transformers-based language models，二是通过 influence data cleansing 去除危险的训练实例。通过在两个 COVID-19 虚假新闻测试集上评估模型，我们强调了此任务的模型泛化能力的重要性。最终模型可以达到高性能和模型泛化性能的平衡。

COVID-19 假新闻检测模型的泛化能力

Model Generalization on COVID-19 Fake News Detection

We review three limitations of BLEU and ROUGE -- the most popular metrics
used to assess reference summaries against hypothesis summaries, come up with
criteria for what a good metric should behave like and propose concrete ways to
use recent Transformers-based Language Models to assess reference summaries
against hypothesis summaries.

该研究回顾了 BLEU 和 ROUGE 两种不足，并提出了衡量文本摘要的好指标需要具备的标准以及使用最近的基于 Transformers 的语言模型来评估参考摘要与假设摘要的具体方法。