We introduce recurrent additive networks (RANs), a new gated rnn which is distinguished by the use of purely additive latent state updates. At every time step, the new state is computed as a gated component-wise