The paper benchmarks several Transformer models [4], to show how these models
can judge sentiment from a news event. This signal can then be used for
downstream modelling and signal identification for commodity trading. We find
that fine-tuned BERT models outperform fine-tuned or vanilla GPT models on this
task. Transformer models have revolutionized the field of natural language
processing (NLP) in recent years, achieving state-of-the-art results on various
tasks such as machine translation, text summarization, question answering, and
natural language generation. Among the most prominent transformer models are
Bidirectional Encoder Representations from Transformers (BERT) and Generative
Pre-trained Transformer (GPT), which differ in their architectures and
objectives.
A CopBERT model training data and process overview is provided. The CopBERT
model outperforms similar domain specific BERT trained models such as FinBERT.
The below confusion matrices show the performance on CopBERT & CopGPT
respectively. We see a ~10 percent increase in f1_score when compare CopBERT vs
GPT4 and 16 percent increase vs CopGPT. Whilst GPT4 is dominant It highlights
the importance of considering alternatives to GPT models for financial
engineering tasks, given risks of hallucinations, and challenges with
interpretability. We unsurprisingly see the larger LLMs outperform the BERT
models, with predictive power. In summary BERT is partially the new XGboost,
what it lacks in predictive power it provides with higher levels of
interpretability. Concluding that BERT models might not be the next XGboost
[2], but represent an interesting alternative for financial engineering tasks,
that require a blend of interpretability and accuracy.

该研究对多个 Transformer 模型进行基准测试，展示了这些模型如何从新闻事件中判断情感。这个信号随后可用于下游建模和商品交易的信号识别。我们发现，在这一任务中，微调的 BERT 模型优于微调或原版的 GPT 模型。研究还提出了 CopBERT 模型，它在域特定的 BERT 训练模型（如 FinBERT）的性能上表现更好。综上所述，BERT 模型可能不会成为下一个 XGboost，但对于需要融合可解释性和准确性的金融工程任务来说，它代表了一个有趣的选择。