This study presents German FinBERT, a novel pre-trained German language model
tailored for financial textual data. The model is trained through a
comprehensive pre-training process, leveraging a substantial corpus comprising
financial reports, ad-hoc announcements and news related to German companies.
The corpus size is comparable to the data sets commonly used for training
standard BERT models. I evaluate the performance of German FinBERT on
downstream tasks, specifically sentiment prediction, topic recognition and
question answering against generic German language models. My results
demonstrate improved performance on finance-specific data, indicating the
efficacy of German FinBERT in capturing domain-specific nuances. The presented
findings suggest that German FinBERT holds promise as a valuable tool for
financial text analysis, potentially benefiting various applications in the
financial domain.

德国 FinBERT 是一种专门针对金融文本数据的新型预训练德语语言模型，通过综合的预训练过程进行训练，利用包括德国公司财务报告、临时公告和新闻在内的大量语料库。评估结果表明，在金融领域数据上，德国 FinBERT 在情绪预测、主题识别和问答等下游任务中表现出了改进的性能，表明其捕捉领域特定细微差别的效果，该研究认为德国 FinBERT 有望成为金融文本分析中有价值的工具，有潜在的应用于金融领域的多个应用场景。

德国 FinBERT：一种德语预训练语言模型

German FinBERT: A German Pre-trained Language Model

Extraction of sentiment signals from news text, stock message boards, and
business reports, for stock movement prediction, has been a rising field of
interest in finance. Building upon past literature, the most recent works
attempt to better capture sentiment from sentences with complex syntactic
structures by introducing aspect-level sentiment classification (ASC). Despite
the growing interest, however, fine-grained sentiment analysis has not been
fully explored in non-English literature due to the shortage of annotated
finance-specific data. Accordingly, it is necessary for non-English languages
to leverage datasets and pre-trained language models (PLM) of different
domains, languages, and tasks to best their performance. To facilitate
finance-specific ASC research in the Korean language, we build KorFinASC, a
Korean aspect-level sentiment classification dataset for finance consisting of
12,613 human-annotated samples, and explore methods of intermediate transfer
learning. Our experiments indicate that past research has been ignorant towards
the potentially wrong knowledge of financial entities encoded during the
training phase, which has overestimated the predictive power of PLMs. In our
work, we use the term "non-stationary knowledge'' to refer to information that
was previously correct but is likely to change, and present "TGT-Masking'', a
novel masking pattern to restrict PLMs from speculating knowledge of the kind.
Finally, through a series of transfer learning with TGT-Masking applied we
improve 22.63% of classification accuracy compared to standalone models on
KorFinASC.

本研究提出了一种基于情感分析和方面级别情感分类的方法，通过中间转移学习，利用跨领域、跨语言和跨任务的数据集和预训练语言模型来提高其在韩语金融数据集上的准确性。