A technical note aiming to offer deeper intuition for the LayerNorm function
common in deep neural networks. LayerNorm is defined relative to a
distinguished 'neural' basis, but it does more than just normalize the
corresponding vector elements. Rather, it implements a composition -- of linear
projection, nonlinear scaling, and then affine transformation -- on input
activation vectors. We develop both a new mathematical expression and geometric
intuition, to make the net effect more transparent. We emphasize that, when
LayerNorm acts on an N-dimensional vector space, all outcomes of LayerNorm lie
within the intersection of an (N-1)-dimensional hyperplane and the interior of
an N-dimensional hyperellipsoid. This intersection is the interior of an
(N-1)-dimensional hyperellipsoid, and typical inputs are mapped near its
surface. We find the direction and length of the principal axes of this
(N-1)-dimensional hyperellipsoid via the eigen-decomposition of a simply
constructed matrix.

一篇技术说明旨在提供对深度神经网络中常见的 LayerNorm 函数更深入的直观理解，通过开发新的数学表达和几何直觉，使其净效应更透明，强调当 LayerNorm 作用于 N 维向量空间时，所有 LayerNorm 的结果位于 (N-1) 维超平面与 N 维超椭球体内部的交集中，该交集是 (N-1) 维超椭球体的内部，而典型输入被映射到其表面附近。我们通过对一个简单构建的矩阵进行特征值分解来找到这个 (N-1) 维超椭球体的主轴方向和长度。