Objective: Text mining of clinical notes embedded in electronic medical
records is increasingly used to extract patient characteristics otherwise not
or only partly available, to assess their association with relevant health
outcomes. As manual data labeling needed to develop text mining models is
resource intensive, we investigated whether off-the-shelf text mining models
developed at external institutions, together with limited within-institution
labeled data, could be used to reliably extract study variables to conduct
association studies.
Materials and Methods: We developed multiple text mining models on different
combinations of within-institution and external-institution data to extract
social factors from discharge reports of intensive care patients. Subsequently,
we assessed the associations between social factors and having a
do-not-resuscitate/intubate code. Results: Important differences were found
between associations based on manually labeled data compared to text-mined
social factors in three out of five cases. Adopting external-institution text
mining models using manually labeled within-institution data resulted in models
with higher F1-scores, but not in meaningfully different associations.
Discussion: While text mining facilitated scaling analyses to larger samples
leading to discovering a larger number of associations, the estimates may be
unreliable. Confirmation is needed with better text mining models, ideally on a
larger manually labeled dataset.
Conclusion: The currently used text mining models were not sufficiently
accurate to be used reliably in an association study. Model adaptation using
within-institution data did not improve the estimates. Further research is
needed to set conditions for reliable use of text mining in medical research.

通过使用来自外部机构的现成文本挖掘模型，与内部标记有限的数据结合使用，可靠地提取研究变量以进行关联研究。我们对在不同数据组合上构建多个文本挖掘模型，并评估了社交因素与是否具有不复苏 / 插管代码之间的关联。虽然文本挖掘可以协助扩展对样本的分析，但估计结果可能是不可靠的。因此，需要进一步研究建立可靠使用文本挖掘进行医学研究的条件。