We present a novel approach incorporating transformer-based language models
into infectious disease modelling. Text-derived features are quantified by
tracking high-density clusters of sentence-level representations of Reddit
posts within specific US states' COVID-19 subreddits. We benchmark these
clustered embedding features against features extracted from other high-quality
datasets. In a threshold-classification task, we show that they outperform all
other feature types at predicting upward trend signals, a significant result
for infectious disease modelling in areas where epidemiological data is
unreliable. Subsequently, in a time-series forecasting task we fully utilise
the predictive power of the caseload and compare the relative strengths of
using different supplementary datasets as covariate feature sets in a
transformer-based time-series model.

通过追踪特定美国州 COVID-19 子版块 Reddit 帖子的句子级表示的高密度聚类，我们将基于转换器的语言模型纳入了传染病建模中，并在许多高质量数据集中进行基准测试。结果表明，除了其他特征类型之外，这些聚类嵌入特征在预测传染病上升趋势信号方面表现最好，对于流行病学数据不可靠的领域具有重要意义。随后，在时间序列预测任务中，我们充分利用了以上模型的预测能力，并比较了使用不同补充数据集作为协变量特征集时，基于转换器的时间序列模型的相对优势。