Comparative reasoning plays a crucial role in text preference prediction;
however, large language models (LLMs) often demonstrate inconsistencies in
their reasoning. While approaches like Chain-of-Thought improve accuracy in
many other settings, they struggle to consistently distinguish the similarities
and differences of complex texts. We introduce SC, a prompting approach that
predicts text preferences by generating structured intermediate comparisons. SC
begins by proposing aspects of comparison, followed by generating textual
comparisons under each aspect. We select consistent comparisons with a pairwise
consistency comparator that ensures each aspect's comparisons clearly
distinguish differences between texts, significantly reducing hallucination and
improving consistency. Our comprehensive evaluations across various NLP tasks,
including summarization, retrieval, and automatic rating, demonstrate that SC
equips LLMs to achieve state-of-the-art performance in text preference
prediction.

SC 通过生成结构化的中间比较来预测文本偏好，选择一致性比较并显著减少虚构和提高一致性，使 LLMs 在文本偏好预测中实现了最先进的性能。

基于何种依据？通过结构化的比较推理预测文本偏好

On What Basis? Predicting Text Preference Via Structured Comparative  Reasoning

In this work, we consider the task of resolving object referents when given a
comparative language description. We present a Multi-view Approach to Grounding
in Context (MAGiC) that leverages transformers to pragmatically reason over
both objects given multiple image views and a language description. In contrast
to past efforts that attempt to connect vision and language for this task
without fully considering the resulting referential context, MAGiC makes use of
the comparative information by jointly reasoning over multiple views of both
object referent candidates and the referring language expression. We present an
analysis demonstrating that comparative reasoning contributes to SOTA
performance on the SNARE object reference task.

本研究采用多视角方法解决了在给定对比性语言描述时解析物体指代的任务，并利用变压器对给定多个图像视图和语言描述进行实用性推理。相比过去的努力仅考虑指代上下文结果而尝试将视觉和语言连接起来，MAGiC 利用对比信息共同推理物体指代候选者和引用语言表达的多个视图。我们进行了分析，证明了相对推理对于在 SNARE 物体指代任务上实现 SOTA 性能的贡献。

对比多视角语言定位

Comparative Multi-View Language Grounding

In this paper, we propose a novel framework to pre-train language models for
enhancing their abilities of comparative reasoning over texts. While recent
research has developed models for NLP tasks that require comparative reasoning,
they suffer from costly manual data labeling and limited generalizability to
different tasks. Our approach involves a scalable method for collecting data
for text-based entity comparison, which leverages both structured and
unstructured data, and the design of three novel pre-training tasks. Evaluation
on a range of downstream tasks including comparative question answering,
question generation, and summarization shows that our pre-training framework
significantly improves the comparative reasoning abilities of language models,
especially under low-resource conditions. This work also releases the first
integrated benchmark for comparative reasoning over texts.

本文提出了一种新的预训练语言模型框架，通过增强其对文本的比较推理能力来提高 NLP 任务的性能，该框架通过收集结构化和非结构化数据设计了三个新的预训练任务，并在比较问答、问句生成和摘要等下游任务中进行了评估，结果显示，我们的预训练框架显著提高了语言模型的比较推理能力，特别是在低资源条件下。此外，本工作还发布了第一个针对文本比较推理的综合基准。