BriefGPT.xyz
Feb, 2023
仅调整规范层的表达能力
The Expressive Power of Tuning Only the Norm Layers
HTML
PDF
Angeliki Giannou, Shashank Rajput, Dimitris Papailiopoulos
TL;DR
本研究探讨了针对正则化层进行精调的可行性,并发现仅针对归一化层的调整能够重构任何目标网络,并验证了这一结论在过度参数化情况下仍然成立。
Abstract
feature normalization
transforms such as Batch and Layer-Normalization have become indispensable ingredients of state-of-the-art deep
neural networks
. Recent studies on
→