A user-generated text on social media enables health workers to keep track of
information, identify possible outbreaks, forecast disease trends, monitor
emergency cases, and ascertain disease awareness and response to official
health correspondence. This exchange of health information on social media has
been regarded as an attempt to enhance public health surveillance (PHS).
Despite its potential, the technology is still in its early stages and is not
ready for widespread application. Advancements in pretrained language models
(PLMs) have facilitated the development of several domain-specific PLMs and a
variety of downstream applications. However, there are no PLMs for social media
tasks involving PHS. We present and release PHS-BERT, a transformer-based PLM,
to identify tasks related to public health surveillance on social media. We
compared and benchmarked the performance of PHS-BERT on 25 datasets from
different social medial platforms related to 7 different PHS tasks. Compared
with existing PLMs that are mainly evaluated on limited tasks, PHS-BERT
achieved state-of-the-art performance on all 25 tested datasets, showing that
our PLM is robust and generalizable in the common PHS tasks. By making PHS-BERT
available, we aim to facilitate the community to reduce the computational cost
and introduce new baselines for future works across various PHS-related tasks.

使用社交媒体上用户生成的文本数据进行公共卫生监测的研究表明，使用预训练语言模型（PLM）的进展促进了针对社交媒体上公共卫生监测任务的新模型的开发。本文介绍了一种名为 PHS-BERT 的 PLM，该模型在公共卫生监测任务上的表现优于现有的 PLMs，并在广泛的数据集上表现出鲁棒性和泛化性。

具有特定领域预训练语言模型的社交媒体公共卫生监测任务基准测试

Benchmarking for Public Health Surveillance tasks on Social Media with a Domain-Specific Pretrained Language Model

Previous research has demonstrated that various properties of infectious
diseases can be inferred from online search behaviour. In this work we use time
series of online search query frequencies to gain insights about the prevalence
of COVID-19 in multiple countries. We first develop unsupervised modelling
techniques based on associated symptom categories identified by the United
Kingdom's National Health Service and Public Health England. We then attempt to
minimise an expected bias in these signals caused by public interest -- as
opposed to infections -- using the proportion of news media coverage devoted to
COVID-19 as a proxy indicator. Our analysis indicates that models based on
online searches precede the reported confirmed cases and deaths by 16.7 (10.2 -
23.2) and 22.1 (17.4 - 26.9) days, respectively. We also investigate transfer
learning techniques for mapping supervised models from countries where the
spread of disease has progressed extensively to countries that are in earlier
phases of their respective epidemic curves. Furthermore, we compare time series
of online search activity against confirmed COVID-19 cases or deaths jointly
across multiple countries, uncovering interesting querying patterns, including
the finding that rarer symptoms are better predictors than common ones.
Finally, we show that web searches improve the short-term forecasting accuracy
of autoregressive models for COVID-19 deaths. Our work provides evidence that
online search data can be used to develop complementary public health
surveillance methods to help inform the COVID-19 response in conjunction with
more established approaches.

本研究使用在线搜索频率的时间序列来洞察多个国家的 COVID-19 盛行情况，并开发了基于联合王国国家卫生服务和公共卫生英格兰公共卫生实验室识别的相关症状类别的无监督建模技术。结果表明，基于在线搜索的模型比由报道的确诊病例和死亡事件组成的做出了更早的预测，并证明了在线搜索数据可以用来开发补充公共卫生监测方法，以帮助应对 COVID-19。