BriefGPT.xyz
Feb, 2024
当表示对齐时:在表示学习动态中的普适性
When Representations Align: Universality in Representation Learning Dynamics
HTML
PDF
Loek van Rossem, Andrew M. Saxe
TL;DR
在复杂的大规模架构中,深度神经网络的表征学习动态可以用编解码映射为任意平滑函数的有效理论来描述,该理论能够概括多种不同激活函数和架构的深度网络的表征学习动态,并展现类似于“丰富”和“懒惰”区域的现象。
Abstract
deep neural networks
come in many sizes and
architectures
. The choice of architecture, in conjunction with the dataset and learning algorithm, is commonly understood to affect the learned neural representations.
→