Large language models (LLMs) with memory are computationally universal.
However, mainstream LLMs are not taking full advantage of memory, and the
designs are heavily influenced by biological brains. Due to their approximate
nature and proneness to the accumulation of errors, conventional neural memory
mechanisms cannot support LLMs to simulate complex reasoning. In this paper, we
seek inspiration from modern computer architectures to augment LLMs with
symbolic memory for complex multi-hop reasoning. Such a symbolic memory
framework is instantiated as an LLM and a set of SQL databases, where the LLM
generates SQL instructions to manipulate the SQL databases. We validate the
effectiveness of the proposed memory framework on a synthetic dataset requiring
complex reasoning. The project website is available at
this https URL .

本文在现代计算机架构的启发下，将符号记忆引入大型语言模型 (LLMs)，并使用 SQL 数据库验证提出的记忆框架的有效性。

ChatDB：用数据库增强 LLMs，作为它们的符号内存

ChatDB: Augmenting LLMs with Databases as Their Symbolic Memory

We propose Neural Entity Reasoner (NE-Reasoner), a framework to introduce
global consistency of recognized entities into Neural Reasoner over Named
Entity Recognition (NER) task. Given an input sentence, the NE-Reasoner layer
can infer over multiple entities to increase the global consistency of output
labels, which then be transfered into entities for the input of next layer.
NE-Reasoner inherits and develops some features from Neural Reasoner 1) a
symbolic memory, allowing it to exchange entities between layers. 2) the
specific interaction-pooling mechanism, allowing it to connect each local word
to multiple global entities, and 3) the deep architecture, allowing it to
bootstrap the recognized entity set from coarse to fine. Like human beings,
NE-Reasoner is able to accommodate ambiguous words and Name Entities that
rarely or never met before. Despite the symbolic information the model
introduced, NE-Reasoner can still be trained effectively in an end-to-end
manner via parameter sharing strategy. NE-Reasoner can outperform conventional
NER models in most cases on both English and Chinese NER datasets. For example,
it achieves state-of-art on CoNLL-2003 English NER dataset.

Neural Entity Reasoner (NE-Reasoner) framework uses symbolic memory and specific interaction-pooling mechanism to introduce global consistency of recognized entities into Neural Reasoner over Named Entity Recognition (NER) task, and can accommodate ambiguous words and rare name entities to outperform conventional NER models in most cases on both English and Chinese NER datasets.