Inspired by the principles of speed reading, we introduce Skim-RNN, a recurrent neural network (RNN) that dynamically decides to update only a small fraction of the hidden state for relatively unimportant input tokens. Skim-RNN gives computational advantage over an RNN that always updates the entire hidden state. Skim-RNN uses the same input and output interfaces as a standard RNN and can be easily used instead of RNNs in existing models. In our experiments, we show that Skim-RNN can achieve significantly reduced computational cost without losing accuracy compared to standard RNNs across five different natural language tasks. In addition, we demonstrate that the trade-off between accuracy and speed of Skim-RNN can be dynamically controlled during inference time in a stable manner. Our analysis also shows that Skim-RNN running on a single CPU offers lower latency compared to standard RNNs on GPUs.

通过运用速读原理，我们提出Skim-RNN，一种递归神经网络（RNN），可以动态地决定仅对相对不重要的输入令牌更新少量隐藏状态，从而让Skim-RNN相对于始终更新整个隐藏状态的RNN获得计算优势。通过五个不同的自然语言任务的实验，我们证明Skim-RNN能够在不损失准确性的情况下显著地减少计算成本。此外，我们还证明Skim-RNN的准确性和速度之间的权衡可以在推理时进行稳定的动态控制。此外，我们的分析还显示，相比于GPU上的标准RNN，单CPU上的Skim-RNN提供更低的延迟。

通过 Skim-RNN 进行神经速读