Objective. Active adverse event surveillance monitors Adverse Drug Events
(ADE) from different data sources, such as electronic health records, medical
literature, social media and search engine logs. Over years, many datasets are
created, and shared tasks are organised to facilitate active adverse event
surveillance. However, most-if not all-datasets or shared tasks focus on
extracting ADEs from a particular type of text. Domain generalisation-the
ability of a machine learning model to perform well on new, unseen domains
(text types)-is under-explored. Given the rapid advancements in natural
language processing, one unanswered question is how far we are from having a
single ADE extraction model that are effective on various types of text, such
as scientific literature and social media posts}. Methods. We contribute to
answering this question by building a multi-domain benchmark for adverse drug
event extraction, which we named MultiADE. The new benchmark comprises several
existing datasets sampled from different text types and our newly created
dataset-CADECv2, which is an extension of CADEC (Karimi, et al., 2015),
covering online posts regarding more diverse drugs than CADEC. Our new dataset
is carefully annotated by human annotators following detailed annotation
guidelines. Conclusion. Our benchmark results show that the generalisation of
the trained models is far from perfect, making it infeasible to be deployed to
process different types of text. In addition, although intermediate transfer
learning is a promising approach to utilising existing resources, further
investigation is needed on methods of domain adaptation, particularly
cost-effective methods to select useful training instances.

在本研究中，通过建立一个名为 MultiADE 的多领域对比基准，我们探索了在不同文本类型中有效地提取不良药物事件的机器学习模型的泛化能力，结果表明训练模型的泛化性还远未达到完美，而进行域适应的方法仍需进一步研究。

MultiADE: 药物不良反应提取的多领域基准

MultiADE: A Multi-domain Benchmark for Adverse Drug Event Extraction

In this paper, we study Chinese Spelling Correction (CSC) as a joint decision
made by two separate models: a language model and an error model. Through
empirical analysis, we find that fine-tuning BERT tends to over-fit the error
model while under-fit the language model, resulting in poor generalization to
out-of-distribution error patterns. Given that BERT is the backbone of most CSC
models, this phenomenon has a significant negative impact. To address this
issue, we are releasing a multi-domain benchmark LEMON, with higher quality and
diversity than existing benchmarks, to allow a comprehensive assessment of the
open domain generalization of CSC models. Then, we demonstrate that a very
simple strategy, randomly masking 20\% non-error tokens from the input sequence
during fine-tuning is sufficient for learning a much better language model
without sacrificing the error model. This technique can be applied to any model
architecture and achieves new state-of-the-art results on SIGHAN, ECSpell, and
LEMON.

通过发现 BERT 对中文拼写纠正的影响，本文提出随机蒙掉输入序列 20％的非错误标记是一种简单有效的提高语言模型和错误模型的技术，它可以被应用于任何模型架构，并在 SIGHAN 等基准测试中取得了新的最优结果

重新思考用于中文拼写纠正的遮盖语言建模

Rethinking Masked Language Modeling for Chinese Spelling Correction

Most existing works on few-shot object detection (FSOD) focus on a setting
where both pre-training and few-shot learning datasets are from a similar
domain. However, few-shot algorithms are important in multiple domains; hence
evaluation needs to reflect the broad applications. We propose a Multi-dOmain
Few-Shot Object Detection (MoFSOD) benchmark consisting of 10 datasets from a
wide range of domains to evaluate FSOD algorithms. We comprehensively analyze
the impacts of freezing layers, different architectures, and different
pre-training datasets on FSOD performance. Our empirical results show several
key factors that have not been explored in previous works: 1) contrary to
previous belief, on a multi-domain benchmark, fine-tuning (FT) is a strong
baseline for FSOD, performing on par or better than the state-of-the-art (SOTA)
algorithms; 2) utilizing FT as the baseline allows us to explore multiple
architectures, and we found them to have a significant impact on down-stream
few-shot tasks, even with similar pre-training performances; 3) by decoupling
pre-training and few-shot learning, MoFSOD allows us to explore the impact of
different pre-training datasets, and the right choice can boost the performance
of the down-stream tasks significantly. Based on these findings, we list
possible avenues of investigation for improving FSOD performance and propose
two simple modifications to existing algorithms that lead to SOTA performance
on the MoFSOD benchmark. The code is available at
this https URL

本文提出了一个基于多领域的数据集的多领域少样本目标检测（MoFSOD）基准测试，以评估少样本算法的性能，并通过冻结层、不同体系结构和不同预训练数据集等方面的实验结果发现了多个影响少样本目标检测性能的关键因素，并提出了两种算法改进方法，在 MoFSOD 基准测试中实现 SOTA 表现。