We view large language models (LLMs) as stochastic \emph{language layers} in
a network, where the learnable parameters are the natural language
\emph{prompts} at each layer. We stack two such layers, feeding the output of
one layer to the next. We call the stacked architecture a \emph{Deep Language
Network} (DLN). We first show how to effectively perform prompt optimization
for a 1-Layer language network (DLN-1). We then show how to train 2-layer DLNs
(DLN-2), where two prompts must be learnt. We consider the output of the first
layer as a latent variable to marginalize, and devise a variational inference
algorithm for joint prompt training. A DLN-2 reaches higher performance than a
single layer, sometimes comparable to few-shot GPT-4 even when each LLM in the
network is smaller and less powerful. The DLN code is open source:
this https URL .

本文提出了 Deep Language Network (DLN) 架构，通过 prompt 优化和变分推理算法，实现了运用较小、较弱 LLM 实现高性能的语言模型，以及进行 few-shot learning 的相关研究。