Large language models~(LLMs) have exhibited impressive performance across NLP
tasks. So far they still face challenges in complex reasoning tasks and can be
sensitive to input context. Despite significant efforts have been invested in
enhancing reasoning process and improving prefix-prompts robustness, the
crucial role of problem context has been overlooked. In this study, we propose
a new approach to improve the mathematical capacities of LLMs, named Problem
Elaboration Prompting~(PEP). Specifically, PEP decomposes and elucidates the
problem context before reasoning, thus enhancing the global context modeling
and reducing the parsing difficulties. Experiments on datasets demonstrate
promising performances on complex reasoning and indicate the beneficial impact
for ill-formed problems. For instance, with the GPT-3.5
model~(\texttt{text-davinci-003}), we observed a 9.93\% improvement with greedy
decoding and 8.80\% improvement with self-consistency on GSM8k compared to the
standard CoT. With ChatGPT~(\texttt{turbo}) and PEP, we achieve SOTA
performances on SVAMP with 86.2\% and GSM8k with 90.98\%.

使用 Problem Elaboration Prompting（PEP）改善大型语言模型（LLMs）的数学能力，在复杂推理和问题背景理解方面取得了有希望的表现。

深思熟虑：问题拓展引导提升大型语言模型的数学推理能力

Look Before You Leap: Problem Elaboration Prompting Improves  Mathematical Reasoning in Large Language Models

The embedding-based representation learning is commonly used in deep learning
recommendation models to map the raw sparse features to dense vectors. The
traditional embedding manner that assigns a uniform size to all features has
two issues. First, the numerous features inevitably lead to a gigantic
embedding table that causes a high memory usage cost. Second, it is likely to
cause the over-fitting problem for those features that do not require too large
representation capacity. Existing works that try to address the problem always
cause a significant drop in recommendation performance or suffers from the
limitation of unaffordable training time cost. In this paper, we proposed a
novel approach, named PEP (short for Plug-in Embedding Pruning), to reduce the
size of the embedding table while avoiding the drop of recommendation accuracy.
PEP prunes embedding parameter where the pruning threshold(s) can be adaptively
learned from data. Therefore we can automatically obtain a mixed-dimension
embedding-scheme by pruning redundant parameters for each feature. PEP is a
general framework that can plug in various base recommendation models.
Extensive experiments demonstrate it can efficiently cut down embedding
parameters and boost the base model's performance. Specifically, it achieves
strong recommendation performance while reducing 97-99% parameters. As for the
computation cost, PEP only brings an additional 20-30% time cost compared with
base models. Codes are available at
this https URL

本篇论文提出一种名为 PEP（即插入嵌入剪枝）的新方法，通过将冗余参数修剪掉提高了基本模型的性能，同时可以自适应地从数据进行修剪阈值的学习，从而自动获得混合维度的嵌入方案，有效地减少了嵌入参数。