When applying automated speech recognition (ASR) for Belgian Dutch (Van Dyck
et al. 2021), the output consists of an unsegmented stream of words, without
any punctuation. A next step is to perform segmentation and insert punctuation,
making the ASR output more readable and easy to manually correct. As far as we
know there is no publicly available punctuation insertion system for Dutch that
functions at a usable level. The model we present here is an extension of the
models of Guhr et al. (2021) for Dutch and is made publicly available. We
trained a sequence classification model, based on the Dutch language model
RobBERT (Delobelle et al. 2020). For every word in the input sequence, the
models predicts a punctuation marker that follows the word. We have also
extended a multilingual model, for cases where the language is unknown or where
code switching applies. When performing the task of segmentation, the
application of the best models onto out of domain test data, a sliding window
of 200 words of the ASR output stream is sent to the classifier, and
segmentation is applied when the system predicts a segmenting punctuation sign
with a ratio above threshold. Results show to be much better than a machine
translation baseline approach.

本文介绍了一个基于 Dutch language model RobBERT 的 sequence classification model ，用于在比利时荷兰语的自动语音识别输出中进行标点符号插入和分割。

使用 Transformers 进行荷兰语标点和分段预测

FullStop:Punctuation and Segmentation Prediction for Dutch with Transformers

We present CoNTACT: a Dutch language model adapted to the domain of COVID-19
tweets. The model was developed by continuing the pre-training phase of RobBERT
(Delobelle, 2020) by using 2.8M Dutch COVID-19 related tweets posted in 2021.
In order to test the performance of the model and compare it to RobBERT, the
two models were tested on two tasks: (1) binary vaccine hesitancy detection and
(2) detection of arguments for vaccine hesitancy. For both tasks, not only
Twitter but also Facebook data was used to show cross-genre performance. In our
experiments, CoNTACT showed statistically significant gains over RobBERT in all
experiments for task 1. For task 2, we observed substantial improvements in
virtually all classes in all experiments. An error analysis indicated that the
domain adaptation yielded better representations of domain-specific
terminology, causing CoNTACT to make more accurate classification decisions.

利用 2021 年发布的 280 万条荷兰语 COVID-19 相关推文进行预训练的 CoNTACT 模型，相比于 RobBERT，在两项任务中（二元疫苗犹豫检测和检测疫苗犹豫的论据）均表现出显著的性能提升，证明了领域自适应的重要性，并且使得 CoNTACT 在领域特定词汇的表示上更加精准。

CoNTACT: 一种适用于荷兰 COVID-19 的 BERT 模型，用于疫苗犹豫和论证检测

CoNTACT: A Dutch COVID-19 Adapted BERT for Vaccine Hesitancy and Argumentation Detection

The transformer-based pre-trained language model BERT has helped to improve
state-of-the-art performance on many natural language processing (NLP) tasks.
Using the same architecture and parameters, we developed and evaluated a
monolingual Dutch BERT model called BERTje. Compared to the multilingual BERT
model, which includes Dutch but is only based on Wikipedia text, BERTje is
based on a large and diverse dataset of 2.4 billion tokens. BERTje consistently
outperforms the equally-sized multilingual BERT model on downstream NLP tasks
(part-of-speech tagging, named-entity recognition, semantic role labeling, and
sentiment analysis). Our pre-trained Dutch BERT model is made available at
this https URL

该研究比较了基于 2.4 亿令牌的广泛数据集训练的荷兰语模型 BERTje 与基于维基百科文本的多语言 BERT 模型，在词性标注、命名实体识别、语义角色标注和情感分析等自然语言处理任务上，BERTje 表现出更好的性能。