We analyze how large language models (LLMs) represent out-of-context words,
investigating their reliance on the given context to capture their semantics.
Our likelihood-guided text perturbations reveal a correlation between token
likelihood and attention values in transformer-based language models. Extensive
experiments reveal that unexpected tokens cause the model to attend less to the
information coming from themselves to compute their representations,
particularly at higher layers. These findings have valuable implications for
assessing the robustness of LLMs in real-world scenarios. Fully reproducible
codebase at this https URL.

我们分析了大型语言模型（LLMs）如何表示上下文之外的单词，研究它们依赖所给上下文来捕捉其语义的情况。我们的文本扰动指导了似然性，揭示了变形器模型中令牌似然性和注意力值之间的相关性。广泛的实验表明，意外的令牌会导致模型不太关注来自自身的信息以计算它们的表示，特别是在较高的层次上。这些发现对于评估 LLMs 在真实世界场景中的鲁棒性具有重要的影响。

变压器中的注意力 - 相似度关系

Attention-likelihood relationship in transformers

We propose an end-to-end-trainable attention module for convolutional neural
network (CNN) architectures built for image classification. The module takes as
input the 2D feature vector maps which form the intermediate representations of
the input image at different stages in the CNN pipeline, and outputs a 2D
matrix of scores for each map. Standard CNN architectures are modified through
the incorporation of this module, and trained under the constraint that a
convex combination of the intermediate 2D feature vectors, as parameterised by
the score matrices, must \textit{alone} be used for classification.
Incentivised to amplify the relevant and suppress the irrelevant or misleading,
the scores thus assume the role of attention values. Our experimental
observations provide clear evidence to this effect: the learned attention maps
neatly highlight the regions of interest while suppressing background clutter.
Consequently, the proposed function is able to bootstrap standard CNN
architectures for the task of image classification, demonstrating superior
generalisation over 6 unseen benchmark datasets. When binarised, our attention
maps outperform other CNN-based attention maps, traditional saliency maps, and
top object proposals for weakly supervised segmentation as demonstrated on the
Object Discovery dataset. We also demonstrate improved robustness against the
fast gradient sign method of adversarial attack.

通过加入可训练的注意力模块，使得卷积神经网络在进行图像分类时能够更好地聚焦于感兴趣的区域，进一步提高了模型的稳健性。