To improve the training efficiency of hierarchical recurrent models without
compromising their performance, we propose a strategy named as `the lower the
simpler', which is to simplify the baseline models by making the lower layers
simpler than the upper layers. We carry out this strategy to simplify two
typical hierarchical recurrent models, namely Hierarchical Recurrent
Encoder-Decoder (HRED) and R-NET, whose basic building block is GRU.
Specifically, we propose Scalar Gated Unit (SGU), which is a simplified variant
of GRU, and use it to replace the GRUs at the middle layers of HRED and R-NET.
Besides, we also use Fixed-size Ordinally-Forgetting Encoding (FOFE), which is
an efficient encoding method without any trainable parameter, to replace the
GRUs at the bottom layers of HRED and R-NET. The experimental results show that
the simplified HRED and the simplified R-NET contain significantly less
trainable parameters, consume significantly less training time, and achieve
slightly better performance than their baseline models.

通过 “越低越简单” 的策略，我们提出了 Scalar Gated Unit 和 Fixed-size Ordinally-Forgetting Encoding 两种方法改进了基于 GRU 的 Hierarchical Recurrent Encoder-Decoder 和 R-NET 模型，显著减少了训练时间和参数数量，且略有性能提升。