In utilizing large language models (LLMs) for mathematical reasoning,
addressing the errors in the reasoning and calculation present in the generated
text by LLMs is a crucial challenge. In this paper, we propose a novel
framework that integrates the Chain-of-Thought (CoT) method with an external
tool (Python REPL). We discovered that by prompting LLMs to generate structured
text in XML-like markup language, we could seamlessly integrate CoT and the
external tool and control the undesired behaviors of LLMs. With our approach,
LLMs can utilize Python computation to rectify errors within CoT. We applied
our method to ChatGPT (GPT-3.5) to solve challenging mathematical problems and
demonstrated that combining CoT and Python REPL through the markup language
enhances the reasoning capability of LLMs. Our approach enables LLMs to write
the markup language and perform advanced mathematical reasoning using only
zero-shot prompting.

利用大型语言模型（LLMs）进行数学推理时，整合 Chain-of-Thought 方法与外部工具（Python REPL）通过标记语言无缝结合，使 LLMs 可以纠正推理中的错误，提高推理能力。

LPML: 数学推理的 LLM-Prompting 标记语言

LPML: LLM-Prompting Markup Language for Mathematical Reasoning

Scientific knowledge is predominantly stored in books and scientific
journals, often in the form of PDFs. However, the PDF format leads to a loss of
semantic information, particularly for mathematical expressions. We propose
Nougat (Neural Optical Understanding for Academic Documents), a Visual
Transformer model that performs an Optical Character Recognition (OCR) task for
processing scientific documents into a markup language, and demonstrate the
effectiveness of our model on a new dataset of scientific documents. The
proposed approach offers a promising solution to enhance the accessibility of
scientific knowledge in the digital age, by bridging the gap between
human-readable documents and machine-readable text. We release the models and
code to accelerate future work on scientific text recognition.

我们提出了一种名为 Nougat 的神经光学理解学术文档的模型，它使用光学字符识别（OCR）将科学文档转换为标记语言。该方法有效地提高了数字时代科学知识的可访问性，将人类可读文档和机器可读文本之间的鸿沟缩小。我们发布了相关模型和代码以促进科学文本识别的未来工作。