AI has made significant progress in solving math problems, but geometry
problems remain challenging due to their reliance on both text and diagrams. In
the text description, symbolic characters such as "$\triangle$ABC" often serve
as a bridge to connect the corresponding diagram. However, by simply tokenizing
symbolic characters into individual letters (e.g., 'A', 'B' and 'C'), existing
works fail to study them explicitly and thus lose the semantic relationship
with the diagram. In this paper, we develop a symbolic character-aware model to
fully explore the role of these characters in both text and diagram
understanding and optimize the model under a multi-modal reasoning framework.
In the text encoder, we propose merging individual symbolic characters to form
one semantic unit along with geometric information from the corresponding
diagram. For the diagram encoder, we pre-train it under a multi-label
classification framework with the symbolic characters as labels. In addition,
we enhance the geometry diagram understanding ability via a self-supervised
learning method under the masked image modeling auxiliary task. By integrating
the proposed model into a general encoder-decoder pipeline for solving geometry
problems, we demonstrate its superiority on two benchmark datasets, including
GeoQA and Geometry3K, with extensive experiments. Specifically, on GeoQA, the
question-solving accuracy is increased from 60.0\% to 64.1\%, achieving a new
state-of-the-art accuracy; on Geometry3K, we reduce the question average
solving steps from 6.9 down to 6.0 with marginally higher solving accuracy.

通过开发一个符号字符感知模型，该研究旨在完全探索符号字符在文本和图解理解中的角色，并在多模态推理框架下优化模型。该模型在解决几何问题的通用编码器 - 解码器流水线中得到整合，经过大量实验，在两个基准数据集（GeoQA 和 Geometry3K）上展现了卓越性能，极大提高了问题解答的准确率和步骤效率。