Language models have achieved impressive results in natural language
processing tasks, but their ability to perform symbolic operations and
arithmetic operations, remains limited, which attribute to their learn the
rules implicitly from data. We explore how to incorporate compiled neural
networks (CoNNs) which weight is specially designed, into the architecture of
language models to enable the language model trained by gradient to obtain
fully rule comprehension ability. The incorporation of compiled neural networks
offers a promising direction for improving the performance of language models
on compound tasks, particularly in areas that require a deeper comprehension of
abstract rules beyond recognizing patterns in training data. Our method, which
call "Neural Comprehension", helps language models achieve absolute accuracy in
symbolic operations, thereby enhancing their ability for rule reasoning,
symbolic reasoning, and arithmetic reasoning. Our code is publicly available
at: https://github.com/WENGSYX/Neural-Comprehension.

本文介绍了如何在语言模型中引入编译的神经网络（CoNNs），以实现语言模型的规则全面理解能力。该方法称为 “神经理解”，可提高语言模型在符合任务中的性能，在需要超越训练数据模式识别的领域中特别有用。在符号操作中，我们的方法能够提高语言模型的绝对准确性，增强其规则推理、符号推理和算术推理能力。