Large Language Models (LLMs) often make errors when performing numerical
calculations. In contrast to traditional chain-of-thought reasoning, the
program-of-thoughts approach involves generating executable code to solve
problems. By executing this code, it achieves more precise results. Using
generated executable code instead of natural language can reduce computational
errors. However, we observe that when LLMs solve mathematical problems using
code, they tend to generate more incorrect reasoning than when using natural
language. To address this issue, we propose Human-Think Language (HTL), a
straightforward yet highly efficient approach inspired by human coding
practices. The approach first generates problem-solving methods described in
the natural language by the model, then converts them into code, mirroring the
process where people think through the logic in natural language before writing
it as code. Additionally, it utilizes the Proximal Policy Optimization (PPO)
algorithm, enabling it to provide feedback to itself based on the correctness
of mathematical answers, much like humans do. Finally, we introduce a
focus-attention mechanism that masks the question segment, enhancing its
reliance on natural language inference solutions during code generation. We
conduct our experiments without introducing any additional information, and the
results across five mathematical calculation datasets showcase the
effectiveness of our approach. Notably, on the NumGLUE dataset, the
LlaMA-2-7B-based model achieves a superior performance rate (75.1%) compared to
the previous best performance with the LlaMA-2-70B model (74.4%).

使用生成的可执行代码代替自然语言可以减少计算错误。本文介绍了一种通过自然语言生成问题解决方法，并将其转换为代码的方法，以提高大型语言模型在数学问题求解中的准确性。同时引入了近端策略优化算法和注意力机制，通过自我反馈和依赖自然语言推理解决方案来改进模型性能。在五个数学计算数据集上进行了实验证明了该方法的有效性。