Retrieval-augmented language models have exhibited promising performance
across various areas of natural language processing (NLP), including
fact-critical tasks. However, due to the black-box nature of advanced large
language models (LLMs) and the non-retrieval-oriented supervision signal of
specific tasks, the training of retrieval model faces significant challenges
under the setting of black-box LLM. We propose an approach leveraging
Fine-grained Feedback with Reinforcement Retrieval (FFRR) to enhance
fact-checking on news claims by using black-box LLM. FFRR adopts a two-level
strategy to gather fine-grained feedback from the LLM, which serves as a reward
for optimizing the retrieval policy, by rating the retrieved documents based on
the non-retrieval ground truth of the task. We evaluate our model on two public
datasets for real-world news claim verification, and the results demonstrate
that FFRR achieves significant improvements over strong LLM-enabled and non-LLM
baselines.

使用黑盒语言模型的精细反馈和增强检索，增强了新闻索赔事实验证的结果，并在真实世界的数据集上显著改进了强大的语言模型和非语言模型基线。

利用细粒度反馈的强化检索在黑箱 LLM 中验证新闻断言

Reinforcement Retrieval Leveraging Fine-grained Feedback for Fact  Checking News Claims with Black-Box LLM

While large pre-trained language models (LLMs) have shown their impressive
capabilities in various NLP tasks, they are still under-explored in the
misinformation domain. In this paper, we examine LLMs with in-context learning
(ICL) for news claim verification, and find that only with 4-shot demonstration
examples, the performance of several prompting methods can be comparable with
previous supervised models. To further boost performance, we introduce a
Hierarchical Step-by-Step (HiSS) prompting method which directs LLMs to
separate a claim into several subclaims and then verify each of them via
multiple questions-answering steps progressively. Experiment results on two
public misinformation datasets show that HiSS prompting outperforms
state-of-the-art fully-supervised approach and strong few-shot ICL-enabled
baselines.

通过实验结果表明，在虚假信息领域，经过 4 个示范样本，多种提示方法的性能可以与以往的监督模型相媲美。为了进一步提升性能，我们引入了分层逐步（HiSS）提示方法，该方法通过逐步回答多个问题来对声明进行验证，实验结果表明 HiSS 提示方法优于最新全监督方法和强少样本 ICL 基线。

基于 LLM 的层次化逐步提示方法用于新闻声称的事实验证

Towards LLM-based Fact Verification on News Claims with a Hierarchical  Step-by-Step Prompting Method

This work explores the application of textual entailment in news claim
verification and stance prediction using a new corpus in Arabic. The publicly
available corpus comes in two perspectives: a version consisting of 4,547 true
and false claims and a version consisting of 3,786 pairs (claim, evidence). We
describe the methodology for creating the corpus and the annotation process.
Using the introduced corpus, we also develop two machine learning baselines for
two proposed tasks: claim verification and stance prediction. Our best model
utilizes pretraining (BERT) and achieves 76.7 F1 on the stance prediction task
and 64.3 F1 on the claim verification task. Our preliminary experiments shed
some light on the limits of automatic claim verification that relies on claims
text only. Results hint that while the linguistic features and world knowledge
learned during pretraining are useful for stance prediction, such learned
representations from pretraining are insufficient for verifying claims without
access to context or evidence.

该论文探讨了在阿拉伯语新闻断言验证和观点预测中使用文本蕴含的应用，并使用一个新的语料库。作者介绍了语料库的创建方法和注释过程，开发了用于两个提出的任务的两个机器学习基线：断言验证和立场预测。作者的最佳模型利用预训练（BERT），在立场预测任务上实现了 76.7 F1，在断言验证任务上实现了 64.3 F1。结果表明，尽管通过预训练学习到的语言特征和世界知识对于立场预测是有用的，但是这些学习表示对于没有上下文或证据的断言验证是不够的。