Geometry problem solving (GPS) is a challenging mathematical reasoning task requiring multi-modal understanding, fusion and reasoning. Existing neural solvers take GPS as a vision-language task but be short in the representation of geometry diagrams which carry rich and complex layout information. In this paper, we propose a layout-aware neural solver named LANS, integrated with two new modules: multimodal layout-aware pre-trained language model (MLA-PLM) and layout-aware fusion attention (LA-FA). MLA-PLM adopts structural and semantic pre-training (SSP) to implement global relationship modeling, and point matching pre-training (PMP) to achieve alignment between visual points and textual points. LA-FA employs a layout-aware attention mask to realize point-guided cross-modal fusion for further boosting layout awareness of LANS. Extensive experiments on datasets Geometry3K and PGPS9K validate the effectiveness of the layout-aware modules and superior problem solving performance of our LANS solver, over existing symbolic solvers and neural solvers. The code will make public available soon.

本文提出了一种名为LANS的布局感知神经求解器，集成了两个新模块:多模态布局感知预训练语言模型(MLA-PLM)和布局感知融合注意力(LA-FA)，通过全局关系建模和对齐来实现布局感知，并通过布局感知注意力实现点导向的跨模态融合，该求解器在几何数据集上的实验证明了布局感知模块的有效性和超越现有符号求解器和神经求解器的问题求解性能。

LANS: 平面几何问题的布局感知神经求解器