In this work, we propose a novel method named \textbf{Auto}mated Process
Labeling via \textbf{C}onfidence \textbf{V}ariation (\textbf{\textsc{AutoCV}})
to enhance the reasoning capabilities of large language models (LLMs) by
automatically annotating the reasoning steps. Our approach begins by training a
verification model on the correctness of final answers, enabling it to generate
automatic process annotations. This verification model assigns a confidence
score to each reasoning step, indicating the probability of arriving at the
correct final answer from that point onward. We detect relative changes in the
verification's confidence scores across reasoning steps to automatically
annotate the reasoning process. This alleviates the need for numerous manual
annotations or the high computational costs associated with model-induced
annotation approaches. We experimentally validate that the confidence
variations learned by the verification model trained on the final answer
correctness can effectively identify errors in the reasoning steps.
Subsequently, we demonstrate that the process annotations generated by
\textsc{AutoCV} can improve the accuracy of the verification model in selecting
the correct answer from multiple outputs generated by LLMs. Notably, we achieve
substantial improvements across five datasets in mathematics and commonsense
reasoning. The source code of \textsc{AutoCV} is available at
https://github.com/rookie-joe/AUTOCV.

我们提出了一种名为 AutoCV 的新方法，通过自动注释推理步骤来增强大型语言模型的推理能力。我们的方法通过训练一个验证模型来为最终答案的正确性评估生成自动过程注释，并且通过检测验证模型在推理步骤中的置信度变化来自动注释推理过程。我们的实验证明，验证模型学习到的置信度变化可以有效地识别推理步骤中的错误，并且通过 AutoCV 生成的过程注释可以提高验证模型在多个输出中选择正确答案的准确性。

AutoCV: 通过置信度变化实现自动过程标注的推理增强

AutoCV: Empowering Reasoning with Automated Process Labeling via  Confidence Variation

Due to the prohibitively high cost of creating error correction datasets,
most Factual Claim Correction methods rely on a powerful verification model to
guide the correction process. This leads to a significant drop in performance
in domains like Scientific Claim Correction, where good verification models do
not always exist. In this work, we introduce a claim correction system that
makes no domain assumptions and does not require a verifier but is able to
outperform existing methods by an order of magnitude -- achieving 94%
correction accuracy on the SciFact dataset, and 62.5% on the SciFact-Open
dataset, compared to the next best methods 0.5% and 1.50% respectively. Our
method leverages the power of prompting with LLMs during training to create a
richly annotated dataset that can be used for fully supervised training and
regularization. We additionally use a claim-aware decoding procedure to improve
the quality of corrected claims. Our method is competitive with the very LLM
that was used to generate the annotated dataset -- with GPT3.5 achieving 89.5%
and 60% correction accuracy on SciFact and SciFact-Open, despite using 1250
times as many parameters as our model.

本文介绍了一个声明校正系统，该系统不需要验证器，并采用了 LLMs 提示和声明感知解码程序进行培训来创建可用于完全监督培训和正则化的丰富注释数据集，在科学事实数据集上可实现 94％纠正准确率。

学生成为大师：匹配 GPT3 进行科学事实错误校正

The student becomes the master: Matching GPT3 on Scientific Factual  Error Correction

Knowing when an output can be trusted is critical for reliably using face
recognition systems. While there has been enormous effort in recent research on
improving face verification performance, understanding when a model's
predictions should or should not be trusted has received far less attention.
Our goal is to assign a confidence score for a face image that reflects its
quality in terms of recognizable information. To this end, we propose a method
for generating image quality training data automatically from 'mated-pairs' of
face images, and use the generated data to train a lightweight Predictive
Confidence Network, termed as PCNet, for estimating the confidence score of a
face image. We systematically evaluate the usefulness of PCNet with its error
versus reject performance, and demonstrate that it can be universally paired
with and improve the robustness of any verification model. We describe three
use cases on the public IJB-C face verification benchmark: (i) to improve 1:1
image-based verification error rates by rejecting low-quality face images; (ii)
to improve quality score based fusion performance on the 1:1 set-based
verification benchmark; and (iii) its use as a quality measure for selecting
high quality (unblurred, good lighting, more frontal) faces from a collection,
e.g. for automatic enrolment or display.

本研究旨在提出一种基于 PCNet 的人脸图像质量评估方法，通过自动生成图像质量训练数据，实现生成可靠的人脸图像的置信度得分，进而提高任何人脸验证模型的识别准确度，以达到三个应用目的：筛选高质量人脸进行自动注册或显示；通过拒绝低质量人脸提高 1:1 基于图像的验证错误率；在 1:1 集合验证基准测试中，提高基于质量分数的融合效果。