While Large Language Models (LLMs) have demonstrated proficiency in handling
complex queries, much of the past work has depended on extensively annotated
datasets by human experts. However, this reliance on fully-supervised
annotations poses scalability challenges, particularly as models and data
requirements grow. To mitigate this, we explore the potential of enhancing
LLMs' reasoning abilities with minimal human supervision. In this work, we
introduce self-reinforcement, which begins with Supervised Fine-Tuning (SFT) of
the model using a small collection of annotated questions. Then it iteratively
improves LLMs by learning from the differences in responses from the SFT and
unfinetuned models on unlabeled questions. Our approach provides an efficient
approach without relying heavily on extensive human-annotated explanations.
However, current reasoning benchmarks typically only include golden-reference
answers or rationales. Therefore, we present \textsc{PuzzleBen}, a weakly
supervised benchmark that comprises 25,147 complex questions, answers, and
human-generated rationales across various domains, such as brainteasers,
puzzles, riddles, parajumbles, and critical reasoning tasks. A unique aspect of
our dataset is the inclusion of 10,000 unannotated questions, enabling us to
explore utilizing fewer supersized data to boost LLMs' inference capabilities.
Our experiments underscore the significance of \textsc{PuzzleBen}, as well as
the effectiveness of our methodology as a promising direction in future
endeavors. Our dataset and code will be published soon on \texttt{Anonymity
Link}.

利用自我强化机制提升大型语言模型（LLMs）的推理能力，并通过一个弱监督基准测试集 	extsc {PuzzleBen}，含有 25,147 个复杂问题以及人工生成的解释，展示了利用较少的监督数据加强 LLMs 推理能力的有效方法。