Large language models (LLMs) use pretraining to predict the subsequent word;
however, their expansion requires significant computing resources. Numerous big
tech companies and research institutes have developed multilingual LLMs (MLLMs)
to meet current demands, overlooking less-resourced languages (LRLs). This
study proposed three strategies to enhance the performance of LRLs based on the
publicly available MLLMs. First, the MLLM vocabularies of LRLs were expanded to
enhance expressiveness. Second, bilingual data were used for pretraining to
align the high- and less-resourced languages. Third, a high-quality small-scale
instruction dataset was constructed and instruction-tuning was performed to
augment the LRL. The experiments employed the Llama2 model and Korean was used
as the LRL, which was quantitatively evaluated against other developed LLMs
across eight tasks. Furthermore, a qualitative assessment was performed based
on human evaluation and GPT4. Experimental results showed that our proposed
Bllossom model exhibited superior performance in qualitative analyses compared
to previously proposed Korean monolingual models.

本研究提出了三种策略以增强不太具备资源的语言在大型语言模型中的表现：扩展词汇表、使用双语数据进行预训练以对齐高资源语言和低资源语言、构建高质量的小规模指令数据集并进行指令微调。通过对比八个任务的其他大型语言模型，在质量分析中，我们的提出的 Bllossom 模型表现出优异的性能。

优化多语言大型语言模型的语言增强：以韩文为例的案例研究

Optimizing Language Augmentation for Multilingual Large Language Models:  A Case Study on Korean

In the era of digital healthcare, the huge volumes of textual information
generated every day in hospitals constitute an essential but underused asset
that could be exploited with task-specific, fine-tuned biomedical language
representation models, improving patient care and management. For such
specialized domains, previous research has shown that fine-tuning models
stemming from broad-coverage checkpoints can largely benefit additional
training rounds over large-scale in-domain resources. However, these resources
are often unreachable for less-resourced languages like Italian, preventing
local medical institutions to employ in-domain adaptation. In order to reduce
this gap, our work investigates two accessible approaches to derive biomedical
language models in languages other than English, taking Italian as a concrete
use-case: one based on neural machine translation of English resources,
favoring quantity over quality; the other based on a high-grade, narrow-scoped
corpus natively written in Italian, thus preferring quality over quantity. Our
study shows that data quantity is a harder constraint than data quality for
biomedical adaptation, but the concatenation of high-quality data can improve
model performance even when dealing with relatively size-limited corpora. The
models published from our investigations have the potential to unlock important
research opportunities for Italian hospitals and academia. Finally, the set of
lessons learned from the study constitutes valuable insights towards a solution
to build biomedical language models that are generalizable to other
less-resourced languages and different domain settings.

本文旨在探究两种途径从英语语料库中翻译生成的语言格式数据和本地化的语料库数据来建立生物医学语言模型，以应用在医疗领域中以提高患者护理和管理（本研究以意大利为例）。通过实验发现，在相对较小的语料库情况下，高质量数据的合并可以提高模型性能，但数据量的限制更为严格；本文的研究成果有望为意大利医院和学术界提供发展机遇，并为如何构建通用于其他不太具备资源的语言和不同领域设置的生物医学语言模型提供有价值的见解。

基于 Transformer 的生物医学语言模型的领域内适应性本地化

Localising In-Domain Adaptation of Transformer-Based Biomedical Language Models

Korean is often referred to as a low-resource language in the research
community. While this claim is partially true, it is also because the
availability of resources is inadequately advertised and curated. This work
curates and reviews a list of Korean corpora, first describing
institution-level resource development, then further iterate through a list of
current open datasets for different types of tasks. We then propose a direction
on how open-source dataset construction and releases should be done for
less-resourced languages to promote research.

本文介绍了韩语语料库的资源开发、开放数据集以及为促进低资源语言的研究提出了开源数据集的构建和发布方向。

开放韩国语料库：实用报告

Open Korean Corpora: A Practical Report

Much work in Natural Language Processing (NLP) has been for resource-rich
languages, making generalization to new, less-resourced languages challenging.
We present two approaches for improving generalization to low-resourced
languages by adapting continuous word representations using linguistically
motivated subword units: phonemes, morphemes and graphemes. Our method requires
neither parallel corpora nor bilingual dictionaries and provides a significant
gain in performance over previous methods relying on these resources. We
demonstrate the effectiveness of our approaches on Named Entity Recognition for
four languages, namely Uyghur, Turkish, Bengali and Hindi, of which Uyghur and
Bengali are low resource languages, and also perform experiments on Machine
Translation. Exploiting subwords with transfer learning gives us a boost of
+15.2 NER F1 for Uyghur and +9.7 F1 for Bengali. We also show improvements in
the monolingual setting where we achieve (avg.) +3 F1 and (avg.) +1.35 BLEU.

使用语言学驱动的子词单元：音素、词素和字形，提高低资源语言下的自然语言处理、实体名称识别和机器翻译表现。