BriefGPT.xyz
Nov, 2015
ACDC:结构化高效的线性层
ACDC: A Structured Efficient Linear Layer
HTML
PDF
Marcin Moczulski, Misha Denil, Jeremy Appleyard, Nando de Freitas
TL;DR
本文提出了一种新的深度完全连接神经网络模块,由对角矩阵和离散余弦变换组成,并说明了如何将ACDC层级联近似于线性层,实验表明可成功地与卷积神经网络相结合,同时提出了该模块与傅里叶光学的联系。
Abstract
The
linear layer
is one of the most pervasive modules in
deep learning
representations. However, it requires $O(N^2)$ parameters and $O(N^2)$ operations. These costs can be prohibitive in mobile applications or p
→