Existing work on understanding deep learning often employs measures that compress all data-dependent information into a few numbers. In this work, we adopt a perspective based on the role of individual examples. We introduce a measure of the computational difficulty of making a prediction for a given input: the (effective) prediction depth. Our extensive investigation reveals surprising yet simple relationships between the prediction depth of a given input and the model's uncertainty, confidence, accuracy and speed of learning for that data point. We further categorize difficult examples into three interpretable groups, demonstrate how these groups are processed differently inside deep models and showcase how this understanding allows us to improve prediction accuracy. Insights from our study lead to a coherent view of a number of separately reported phenomena in the literature: early layers generalize while later layers memorize; early layers converge faster and networks learn easy data and simple functions first.

本研究采用基于个例角度的视角，引入了计算预测输入的困难程度的度量-（有效的）预测深度，发现了一些有意为之而意外的、简单的关系，进一步将困难例子分为三个可解释的组，并展示了这种理解如何提高了预测准确率，揭示了文献中一些分开探讨的现象的连贯视图。

从示例难度的角度看深度学习