We present BARTpho with two versions, BARTpho-syllable and BARTpho-word,
which are the first public large-scale monolingual sequence-to-sequence models
pre-trained for Vietnamese. BARTpho uses the "large" architecture and the
pre-training scheme of the sequence-to-sequence denoising autoencoder BART,
thus it is especially suitable for generative NLP tasks. We conduct experiments
to compare our BARTpho with its competitor mBART on a downstream task of
Vietnamese text summarization and show that: in both automatic and human
evaluations, BARTpho outperforms the strong baseline mBART and improves the
state-of-the-art. We further evaluate and compare BARTpho and mBART on the
Vietnamese capitalization and punctuation restoration tasks and also find that
BARTpho is more effective than mBART on these two tasks. We publicly release
BARTpho to facilitate future research and applications of generative Vietnamese
NLP tasks. Our BARTpho models are available at
this https URL

本文介绍了两个版本的 BARTpho 模型：BARTpho-syllable 和 BARTpho-word，这两个模型是首个由越南语言训练的公共大规模单语序列到序列模型，使用 BART 的预训练机制，适用于生成式自然语言处理任务。在越南文本摘要等下游任务中，BARTpho 比强基线 mBART 表现更好，进一步在越南大写字母和标点符号恢复任务中表现更为有效。