This study compares the performance of (1) fine-tuned models and (2)
extremely large language models on the task of check-worthy claim detection.
For the purpose of the comparison we composed a multilingual and multi-topical
dataset comprising texts of various sources and styles. Building on this, we
performed a benchmark analysis to determine the most general multilingual and
multi-topical claim detector.
We chose three state-of-the-art models in the check-worthy claim detection
task and fine-tuned them. Furthermore, we selected three state-of-the-art
extremely large language models without any fine-tuning. We made modifications
to the models to adapt them for multilingual settings and through extensive
experimentation and evaluation. We assessed the performance of all the models
in terms of accuracy, recall, and F1-score in in-domain and cross-domain
scenarios. Our results demonstrate that despite the technological progress in
the area of natural language processing, the models fine-tuned for the task of
check-worthy claim detection still outperform the zero-shot approaches in a
cross-domain settings.

本研究比较了经过调整的模型和极大语言模型在可检测可信度主张的任务中的性能。通过使用包含不同来源和风格的文本构建了一个多语种和多主题数据集，并基于此进行了基准分析，确定了最通用的多语种和多主题主张检测器。我们选择了三个最先进的模型进行了可检测可信度主张任务的调整，并选择了三个无需任何调整的最先进极大语言模型。通过对模型进行修改以适应多语种环境，并进行了广泛的实验和评估。在域内和跨域情景中，我们评估了所有模型的准确性、召回率和 F1 分数。我们的结果表明，尽管在自然语言处理领域取得了技术进步，但针对可检测可信度主张任务的调整模型在跨域设置中仍然优于零样本方法。