The use of NLP in the realm of financial technology is broad and complex,
with applications ranging from sentiment analysis and named entity recognition
to question answering. Large Language Models (LLMs) have been shown to be
effective on a variety of tasks; however, no LLM specialized for the financial
domain has been reported in literature. In this work, we present BloombergGPT,
a 50 billion parameter language model that is trained on a wide range of
financial data. We construct a 363 billion token dataset based on Bloomberg's
extensive data sources, perhaps the largest domain-specific dataset yet,
augmented with 345 billion tokens from general purpose datasets. We validate
BloombergGPT on standard LLM benchmarks, open financial benchmarks, and a suite
of internal benchmarks that most accurately reflect our intended usage. Our
mixed dataset training leads to a model that outperforms existing models on
financial tasks by significant margins without sacrificing performance on
general LLM benchmarks. Additionally, we explain our modeling choices, training
process, and evaluation methodology. As a next step, we plan to release
training logs (Chronicles) detailing our experience in training BloombergGPT.

本文介绍了 BloombergGPT， 这是一个在大量金融数据上训练得到的具有 500 亿参数的语言模型。通过使用混合数据集进行训练，我们得到的模型不仅在金融任务上表现出色，还在普遍的 LLM 基准测试上得到了不错的表现，同时也解释了模型构建、训练过程和评估方法。