Numerical reasoning over table-and-text hybrid passages, such as financial
reports, poses significant challenges and has numerous potential applications.
Noise and irrelevant variables in the model input have been a hindrance to its
performance. Additionally, coarse-grained supervision of the whole solution
program has impeded the model's ability to learn the underlying numerical
reasoning process. In this paper, we propose three pretraining tasks that
operate at both the whole program and sub-program level: Variable Integrity
Ranking, which guides the model to focus on useful variables; Variable Operator
Prediction, which decomposes the supervision into fine-grained single operator
prediction; and Variable Keyphrase Masking, which encourages the model to
identify key evidence that sub-programs are derived from. Experimental results
demonstrate the effectiveness of our proposed methods, surpassing
transformer-based model baselines.

本文提出了三个预训练任务，具体包括：变量完整性排名（Variable Integrity Ranking）；变量操作符预测（Variable Operator Prediction）；变量关键词屏蔽（Variable Keyphrase Masking）并应用这些任务对数字推理进行建模，结果表明这种方法比基于 Transformer 的模型胜出，对金融报表等表格和文本混合结构进行数字推理具有较高的效果。