We present a complimentary objective for training recurrent neural networks (RNN) with gating units that helps with regularization and interpreta