The trade-off between expressiveness and interpretability remains a core
challenge when building human-centric predictive models for classification and
decision-making. While symbolic rules offer interpretability, they often lack
expressiveness, whereas neural networks excel in performance but are known for
being black boxes. In this paper, we show a combination of Large Language
Models (LLMs) and symbolic programs can bridge this gap. In the proposed
LLM-based Symbolic Programs (LSPs), the pretrained LLM with natural language
prompts provides a massive set of interpretable modules that can transform raw
input into natural language concepts. Symbolic programs then integrate these
modules into an interpretable decision rule. To train LSPs, we develop a
divide-and-conquer approach to incrementally build the program from scratch,
where the learning process of each step is guided by LLMs. To evaluate the
effectiveness of LSPs in extracting interpretable and accurate knowledge from
data, we introduce IL-Bench, a collection of diverse tasks, including both
synthetic and real-world scenarios across different modalities. Empirical
results demonstrate LSP's superior performance compared to traditional
neurosymbolic programs and vanilla automatic prompt tuning methods. Moreover,
as the knowledge learned by LSP is a combination of natural language
descriptions and symbolic rules, it is easily transferable to humans
(interpretable), and other LLMs, and generalizes well to out-of-distribution
samples.

大型语言模型与符号程序的结合可以在构建人类中心预测模型时解决表达力和解释性之间的平衡问题。

大型语言模型是可解释学习者

Large Language Models are Interpretable Learners

We propose the Neuro-Symbolic Concept Learner (NS-CL), a model that learns
visual concepts, words, and semantic parsing of sentences without explicit
supervision on any of them; instead, our model learns by simply looking at
images and reading paired questions and answers. Our model builds an
object-based scene representation and translates sentences into executable,
symbolic programs. To bridge the learning of two modules, we use a
neuro-symbolic reasoning module that executes these programs on the latent
scene representation. Analogical to human concept learning, the perception
module learns visual concepts based on the language description of the object
being referred to. Meanwhile, the learned visual concepts facilitate learning
new words and parsing new sentences. We use curriculum learning to guide the
searching over the large compositional space of images and language. Extensive
experiments demonstrate the accuracy and efficiency of our model on learning
visual concepts, word representations, and semantic parsing of sentences.
Further, our method allows easy generalization to new object attributes,
compositions, language concepts, scenes and questions, and even new program
domains. It also empowers applications including visual question answering and
bidirectional image-text retrieval.

提出了一种无需显式监督，通过观察图片和配对问答学习视觉概念、单词和句子语义分析的神经符号概念学习器，其通过神经符号推理模块将符号程序执行于潜在场景表示，辅助学习新词并解析新句子，使用课程学习指导对图像和语言的复合空间进行搜索，并进行了广泛实验来证实其在学习视觉概念、单词表示和语义分析方面的准确性和效率。