Recent advancements highlight the success of instruction tuning with large
language models (LLMs) utilizing Chain-of-Thought (CoT) data for mathematical
reasoning tasks. Despite the fine-tuned LLMs, challenges persist, such as
incorrect, missing, and redundant steps in CoT generation leading to
inaccuracies in answer predictions. To alleviate this problem, we propose a
dual instruction tuning strategy to meticulously model mathematical reasoning
from both forward and reverse directions. This involves introducing the
Intermediate Reasoning State Prediction task (forward reasoning) and the
Instruction Reconstruction task (reverse reasoning) to enhance the LLMs'
understanding and execution of instructions. Training instances for these tasks
are constructed based on existing mathematical instruction tuning datasets.
Subsequently, LLMs undergo multi-task fine-tuning using both existing
mathematical instructions and the newly created data. Comprehensive experiments
validate the effectiveness and domain generalization of the dual instruction
tuning strategy across various mathematical reasoning tasks.

介绍了利用大型语言模型进行指令调优的最新进展，提出了一种双向指令调优策略，通过前向和后向推理任务来改善语言模型对数学推理的理解与执行能力。通过广泛实验证明，这种策略适用于多种数学推理任务并具有领域通用性。