A core component present in many successful neural network architectures, is
an MLP block of two fully connected layers with a non-linear activation in
between. An intriguing phenomenon observed empirically, including in
transformer architectures, is that, after training, the activations in the
hidden layer of this MLP block tend to be extremely sparse on any given input.
Unlike traditional forms of sparsity, where there are neurons/weights which can
be deleted from the network, this form of {\em dynamic} activation sparsity
appears to be harder to exploit to get more efficient networks. Motivated by
this we initiate a formal study of PAC learnability of MLP layers that exhibit
activation sparsity. We present a variety of results showing that such classes
of functions do lead to provable computational and statistical advantages over
their non-sparse counterparts. Our hope is that a better theoretical
understanding of {\em sparsely activated} networks would lead to methods that
can exploit activation sparsity in practice.

许多成功的神经网络结构中的核心组件是一个具有非线性激活函数的两个全连接层的 MLP 块。我们在本文中对展示出激活稀疏性的 MLP 层的 PAC 可学习性进行了形式化研究，并呈现了多种实验结果，表明这类函数相对于非稀疏的对应物具有计算和统计上的优势。我们希望对 “激活稀疏” 的网络有更好的理论认识，以便能够在实践中利用激活稀疏性。

稀疏激活下的神经网络学习

Learning Neural Networks with Sparse Activations

Going beyond mimicking limited human experiences, recent studies show initial
evidence that, like humans, large language models (LLMs) are capable of
improving their abilities purely by self-correction, i.e., correcting previous
responses through self-examination, in certain circumstances. Nevertheless,
little is known about how such capabilities arise. In this work, based on a
simplified setup akin to an alignment task, we theoretically analyze
self-correction from an in-context learning perspective, showing that when LLMs
give relatively accurate self-examinations as rewards, they are capable of
refining responses in an in-context way. Notably, going beyond previous
theories on over-simplified linear transformers, our theoretical construction
underpins the roles of several key designs of realistic transformers for
self-correction: softmax attention, multi-head attention, and the MLP block. We
validate these findings extensively on synthetic datasets. Inspired by these
findings, we also illustrate novel applications of self-correction, such as
defending against LLM jailbreaks, where a simple self-correction step does make
a large difference. We believe that these findings will inspire further
research on understanding, exploiting, and enhancing self-correction for
building better foundation models.

基于一种类似于对齐任务的简化设置，我们从上下文学习的角度对自我纠正进行理论分析，显示出大语言模型通过给予相对准确的自我检查作为奖励，能够以上下文的方式改进响应。我们的理论构建突破了之前关于过于简化的线性变换器的理论，阐述了现实变换器的几个关键设计在自我纠正中的作用：softmax 关注、多头关注和 MLP 块。经过广泛的合成数据集验证，我们进一步阐述了自我纠正的新应用，例如抵制大型语言模型越狱，其中简单的自我纠正步骤确实产生了很大的差异。我们相信这些发现将激发进一步研究自我纠正的理解、利用和增强，以构建更好的基础模型。