We present CoNTACT: a Dutch language model adapted to the domain of COVID-19
tweets. The model was developed by continuing the pre-training phase of RobBERT
(Delobelle, 2020) by using 2.8M Dutch COVID-19 related tweets posted in 2021.
In order to test the performance of the model and compare it to RobBERT, the
two models were tested on two tasks: (1) binary vaccine hesitancy detection and
(2) detection of arguments for vaccine hesitancy. For both tasks, not only
Twitter but also Facebook data was used to show cross-genre performance. In our
experiments, CoNTACT showed statistically significant gains over RobBERT in all
experiments for task 1. For task 2, we observed substantial improvements in
virtually all classes in all experiments. An error analysis indicated that the
domain adaptation yielded better representations of domain-specific
terminology, causing CoNTACT to make more accurate classification decisions.

利用 2021 年发布的 280 万条荷兰语 COVID-19 相关推文进行预训练的 CoNTACT 模型，相比于 RobBERT，在两项任务中（二元疫苗犹豫检测和检测疫苗犹豫的论据）均表现出显著的性能提升，证明了领域自适应的重要性，并且使得 CoNTACT 在领域特定词汇的表示上更加精准。