By now there is substantial evidence that deep learning models learn certain
human-interpretable features as part of their internal representations of data.
As having the right (or wrong) concepts is critical to trustworthy machine
learning systems, it is natural to ask which inputs from the model's original
training set were most important for learning a concept at a given layer. To
answer this, we combine data attribution methods with methods for probing the
concepts learned by a model. Training network and probe ensembles for two
concept datasets on a range of network layers, we use the recently developed
TRAK method for large-scale data attribution. We find some evidence for
convergence, where removing the 10,000 top attributing images for a concept and
retraining the model does not change the location of the concept in the network
nor the probing sparsity of the concept. This suggests that rather than being
highly dependent on a few specific examples, the features that inform the
development of a concept are spread in a more diffuse manner across its
exemplars, implying robustness in concept formation.

通过深度学习模型和数据归因方法，在多层网络上训练网络和探针集合，研究了概念形成的可靠性和内部表示中的人类可解释特征。

神经网络中将学习到的概念归因于训练数据

Attributing Learned Concepts in Neural Networks to Training Data

For the convolutional neural network (CNN) used for pattern classification,
the training loss function is usually applied to the final output of the
network, except for some regularization constraints on the network parameters.
However, with the increasing of the number of network layers, the influence of
the loss function on the network front layers gradually decreases, and the
network parameters tend to fall into local optimization. At the same time, it
is found that the trained network has significant information redundancy at all
stages of features, which reduces the effectiveness of feature mapping at all
stages and is not conducive to the change of the subsequent parameters of the
network in the direction of optimality. Therefore, it is possible to obtain a
more optimized solution of the network and further improve the classification
accuracy of the network by designing a loss function for restraining the front
stage features and eliminating the information redundancy of the front stage
features .For CNN, this article proposes a multi-stage feature decorrelation
loss (MFD Loss), which refines effective features and eliminates information
redundancy by constraining the correlation of features at all stages.
Considering that there are many layers in CNN, through experimental comparison
and analysis, MFD Loss acts on multiple front layers of CNN, constrains the
output features of each layer and each channel, and performs supervision
training jointly with classification loss function during network training.
Compared with the single Softmax Loss supervised learning, the experiments on
several commonly used datasets on several typical CNNs prove that the
classification performance of Softmax Loss+MFD Loss is significantly better.
Meanwhile, the comparison experiments before and after the combination of MFD
Loss and some other typical loss functions verify its good universality.

为了优化卷积神经网络的训练结果，本文提出了一种多阶段特征去相关损失函数（MFD Loss），通过约束各个阶段的特征相关性，优化特征映射，进一步提高分类准确性。与单一 Softmax Loss 相比，使用 Softmax Loss+MFD Loss 的分类性能明显更好，并且实验证明 MFD Loss 与其他典型损失函数的组合效果也很好。

改进 CNN 分类性能的多阶段特征去相关约束

Multi-stage feature decorrelation constraints for improving CNN  classification performance

Probing experiments investigate the extent to which neural representations
make properties -- like part-of-speech -- predictable. One suggests that a
representation encodes a property if probing that representation produces
higher accuracy than probing a baseline representation like non-contextual word
embeddings. Instead of using baselines as a point of comparison, we're
interested in measuring information that is contained in the representation but
not in the baseline. For example, current methods can detect when a
representation is more useful than the word identity (a baseline) for
predicting part-of-speech; however, they cannot detect when the representation
is predictive of just the aspects of part-of-speech not explainable by the word
identity. In this work, we extend a theory of usable information called
$\mathcal{V}$-information and propose conditional probing, which explicitly
conditions on the information in the baseline. In a case study, we find that
after conditioning on non-contextual word embeddings, properties like
part-of-speech are accessible at deeper layers of a network than previously
thought.

研究神经表示在探测实验中的有效性并提出一种条件探测的理论，通过该理论研究发现在条件化非上下文单词嵌入后，部分性质如词性是可以在深层网络中被探测的。

条件探测：在基线之外测量可用信息

Conditional probing: measuring usable information beyond a baseline

Crowd counting presents enormous challenges in the form of large variation in
scales within images and across the dataset. These issues are further
exacerbated in highly congested scenes. Approaches based on straightforward
fusion of multi-scale features from a deep network seem to be obvious solutions
to this problem. However, these fusion approaches do not yield significant
improvements in the case of crowd counting in congested scenes. This is usually
due to their limited abilities in effectively combining the multi-scale
features for problems like crowd counting. To overcome this, we focus on how to
efficiently leverage information present in different layers of the network.
Specifically, we present a network that involves: (i) a multi-level bottom-top
and top-bottom fusion (MBTTBF) method to combine information from shallower to
deeper layers and vice versa at multiple levels, (ii) scale complementary
feature extraction blocks (SCFB) involving cross-scale residual functions to
explicitly enable flow of complementary features from adjacent conv layers
along the fusion paths. Furthermore, in order to increase the effectiveness of
the multi-scale fusion, we employ a principled way of generating scale-aware
ground-truth density maps for training. Experiments conducted on three datasets
that contain highly congested scenes (ShanghaiTech, UCF_CC_50, and UCF-QNRF)
demonstrate that the proposed method is able to outperform several recent
methods in all the datasets.

针对高拥挤场景下的人群计数问题，我们提出了一种网络结构，利用多级底部 - 顶部和顶部 - 底部融合方法、跨度残差函数和精心生成的尺度感知的密度地图等技术，有效地解决了不同层次的信息整合问题，大大提高了计数的精度。