While Large Language Models (LLMs) have demonstrated proficiency in handling
complex queries, much of the past work has depended on extensively annotated
datasets by human experts. However, this reliance on fully-supervised
annotations poses scalability challenges, particularly as models and data
requirements grow. To mitigate this, we explore the potential of enhancing
LLMs' reasoning abilities with minimal human supervision. In this work, we
introduce self-reinforcement, which begins with Supervised Fine-Tuning (SFT) of
the model using a small collection of annotated questions. Then it iteratively
improves LLMs by learning from the differences in responses from the SFT and
unfinetuned models on unlabeled questions. Our approach provides an efficient
approach without relying heavily on extensive human-annotated explanations.
However, current reasoning benchmarks typically only include golden-reference
answers or rationales. Therefore, we present \textsc{PuzzleBen}, a weakly
supervised benchmark that comprises 25,147 complex questions, answers, and
human-generated rationales across various domains, such as brainteasers,
puzzles, riddles, parajumbles, and critical reasoning tasks. A unique aspect of
our dataset is the inclusion of 10,000 unannotated questions, enabling us to
explore utilizing fewer supersized data to boost LLMs' inference capabilities.
Our experiments underscore the significance of \textsc{PuzzleBen}, as well as
the effectiveness of our methodology as a promising direction in future
endeavors. Our dataset and code will be published soon on \texttt{Anonymity
Link}.

利用自我强化机制提升大型语言模型（LLMs）的推理能力，并通过一个弱监督基准测试集 	extsc {PuzzleBen}，含有 25,147 个复杂问题以及人工生成的解释，展示了利用较少的监督数据加强 LLMs 推理能力的有效方法。

优化语言模型的推理能力的弱监督方法

Optimizing Language Model's Reasoning Abilities with Weak Supervision

Current privacy research on large language models (LLMs) primarily focuses on
the issue of extracting memorized training data. At the same time, models'
inference capabilities have increased drastically. This raises the key question
of whether current LLMs could violate individuals' privacy by inferring
personal attributes from text given at inference time. In this work, we present
the first comprehensive study on the capabilities of pretrained LLMs to infer
personal attributes from text. We construct a dataset consisting of real Reddit
profiles, and show that current LLMs can infer a wide range of personal
attributes (e.g., location, income, sex), achieving up to $85\%$ top-1 and
$95.8\%$ top-3 accuracy at a fraction of the cost ($100\times$) and time
($240\times$) required by humans. As people increasingly interact with
LLM-powered chatbots across all aspects of life, we also explore the emerging
threat of privacy-invasive chatbots trying to extract personal information
through seemingly benign questions. Finally, we show that common mitigations,
i.e., text anonymization and model alignment, are currently ineffective at
protecting user privacy against LLM inference. Our findings highlight that
current LLMs can infer personal data at a previously unattainable scale. In the
absence of working defenses, we advocate for a broader discussion around LLM
privacy implications beyond memorization, striving for a wider privacy
protection.

当前隐私研究主要集中在大型语言模型（LLM）提取训练数据的问题上。与此同时，模型的推论能力已大幅增强，这引发了一个关键问题：当前的 LLM 是否能通过推断来侵犯个人的隐私。在本研究中，我们提出了关于预训练 LLM 从文本中推断个人属性能力的首个全面研究。我们构建了一个由真实 Reddit 个人资料组成的数据集，并展示出当前 LLM 能够推断广泛的个人属性（如地点、收入、性别），在成本（人类所需的 1%）和时间（人类所需的 2.4%）上达到了高达 85% 的 top-1 准确率和 95.8% 的 top-3 准确率。由于人们越来越多地与 LLM 驱动的聊天机器人进行各个方面的互动，我们还探讨了通过似乎无害的问题来提取个人信息的侵犯隐私聊天机器人的新威胁。最后，我们证明了普遍采用的缓解措施，即文本匿名化和模型对齐，对于保护用户隐私免受 LLM 推断攻击是无效的。我们的研究结果表明，当前的 LLM 能够以以前无法想象的规模推断出个人数据。在缺乏有效防御措施的情况下，我们主张就 LLM 隐私影响展开更广泛的讨论，力求实现更广泛的隐私保护。

超越死记硬背：利用大型语言模型的推理违反隐私

Beyond Memorization: Violating Privacy Via Inference with Large Language  Models

Recently, the community has achieved substantial progress on many commonsense
reasoning benchmarks. However, it is still unclear what is learned from the
training process: the knowledge, inference capability, or both? We argue that
due to the large scale of commonsense knowledge, it is infeasible to annotate a
large enough training set for each task to cover all commonsense for learning.
Thus we should separate the commonsense knowledge acquisition and inference
over commonsense knowledge as two separate tasks. In this work, we focus on
investigating models' commonsense inference capabilities from two perspectives:
(1) Whether models can know if the knowledge they have is enough to solve the
task; (2) Whether models can develop commonsense inference capabilities that
generalize across commonsense tasks. We first align commonsense tasks with
relevant knowledge from commonsense knowledge bases and ask humans to annotate
whether the knowledge is enough or not. Then, we convert different commonsense
tasks into a unified question answering format to evaluate models'
generalization capabilities. We name the benchmark as Commonsense Inference
with Knowledge-in-the-loop Question Answering (CIKQA).

本文提倡将 commonsense knowledge acquisition 和 inference over commonsense knowledge 视为两个不同的任务，通过人类标注的方式评估模型的 commonsense inference capabilities，提出了一种新的 benchmark——CIKQA。