In recent years, there has been interest in how geometric properties such as intrinsic dimension (ID) of a neural network's hidden representations evolve through its layers, and how such properties are predictive of important model behavior such as generalization ability. However, evidence has begun to emerge that such behavior can change significantly depending on the domain of the network's training data, such as natural versus medical images. Here, we further this inquiry by exploring how the ID of a network's learned representations evolves through its layers, in essence, characterizing how the network successively refines the information content of input data to be used for predictions. Analyzing eleven natural and medical image datasets across six network architectures, we find that the shape of this ID evolution curve differs noticeably between natural and medical image models: medical image models peak in representation ID earlier in the network, implying a difference in the image features and their abstractness that are typically used for downstream tasks in these domains. Additionally, we discover a strong correlation of this peak representation ID with the ID of the data in its input space, implying that the intrinsic information content of a model's learned representations is guided by that of the data it was trained on. Overall, our findings emphasize notable discrepancies in network behavior between natural and non-natural imaging domains regarding hidden representation information content, and provide further insights into how a network's learned features are shaped by its training data.

本研究探讨了神经网络的隐藏表征内在维度在不同层次上的演变，揭示了自然影像与医学影像模型之间的显著差异。研究发现医学影像模型在网络较早阶段就达到内在维度的峰值，表明两类数据在特征抽象性上的不同，且这种峰值与输入数据的内在信息内容密切相关。这些发现为神经网络的学习特征如何受训练数据影响提供了新见解。

预处理与压缩：通过内在维度理解成像领域中隐藏表征的优化