Deep neural networks whose parameters are distributed according to typical initialization schemes exhibit undesirable properties that can emerge as the number of layers increases. These issues include a vanishing dependency on the input and a concentration on restrictive families of fu