Historical records in Korea before the 20th century were primarily written in
Hanja, an extinct language based on Chinese characters and not understood by
modern Korean or Chinese speakers. Historians with expertise in this time
period have been analyzing the documents, but that process is very difficult
and time-consuming, and language models would significantly speed up the
process. Toward building and evaluating language models for Hanja, we release
the Hanja Understanding Evaluation dataset consisting of chronological
attribution, topic classification, named entity recognition, and summary
retrieval tasks. We also present BERT-based models continued training on the
two major corpora from the 14th to the 19th centuries: the Annals of the Joseon
Dynasty and Diaries of the Royal Secretariats. We compare the models with
several baselines on all tasks and show there are significant improvements
gained by training on the two corpora. Additionally, we run zero-shot
experiments on the Daily Records of the Royal Court and Important Officials
(DRRI). The DRRI dataset has not been studied much by the historians, and not
at all by the NLP community.

本篇研究旨在建立、评估韩字理解的语言模型，为此发布了包含了年代归属、主题分类、命名实体识别和概要检索任务的 Hanja Understanding Evaluation 数据集。同时，还在朝鲜王朝的官方纪事 Annals 和 Royal Secretariats 的两个重要文献语料库上训练了 BERT 模型，并将它与几个基线模型进行了比较，发现训练在这两个文献上的模型具有显著改进。此外，还在历史学研究中尚未被广泛研究的 DRRI 数据集上进行了零样本实验。