In the era of digital healthcare, the huge volumes of textual information
generated every day in hospitals constitute an essential but underused asset
that could be exploited with task-specific, fine-tuned biomedical language
representation models, improving patient care and management. For such
specialized domains, previous research has shown that fine-tuning models
stemming from broad-coverage checkpoints can largely benefit additional
training rounds over large-scale in-domain resources. However, these resources
are often unreachable for less-resourced languages like Italian, preventing
local medical institutions to employ in-domain adaptation. In order to reduce
this gap, our work investigates two accessible approaches to derive biomedical
language models in languages other than English, taking Italian as a concrete
use-case: one based on neural machine translation of English resources,
favoring quantity over quality; the other based on a high-grade, narrow-scoped
corpus natively written in Italian, thus preferring quality over quantity. Our
study shows that data quantity is a harder constraint than data quality for
biomedical adaptation, but the concatenation of high-quality data can improve
model performance even when dealing with relatively size-limited corpora. The
models published from our investigations have the potential to unlock important
research opportunities for Italian hospitals and academia. Finally, the set of
lessons learned from the study constitutes valuable insights towards a solution
to build biomedical language models that are generalizable to other
less-resourced languages and different domain settings.

本文旨在探究两种途径从英语语料库中翻译生成的语言格式数据和本地化的语料库数据来建立生物医学语言模型，以应用在医疗领域中以提高患者护理和管理（本研究以意大利为例）。通过实验发现，在相对较小的语料库情况下，高质量数据的合并可以提高模型性能，但数据量的限制更为严格；本文的研究成果有望为意大利医院和学术界提供发展机遇，并为如何构建通用于其他不太具备资源的语言和不同领域设置的生物医学语言模型提供有价值的见解。