Can multilayer neural networks -- typically constructed as highly complex
structures with many nonlinearly activated neurons across layers -- behave in a
non-trivial way that yet simplifies away a major part of their complexities? In
this work, we uncover a phenomenon in which the beha