Sequential processes in real-world often carry a combination of simple
subsystems that interact with each other in certain forms. Learning such a
modular structure can often improve the robustness against environmental
changes. In this paper, we propose recurrent independent Grid LSTM (RigLSTM),
composed of a group of independent LSTM cells that cooperate with each other,
for exploiting the underlying modular structure of the target task. Our model
adopts cell selection, input feature selection, hidden state selection, and
soft state updating to achieve a better generalization ability on the basis of
the recent Grid LSTM for the tasks where some factors differ between training
and evaluation. Specifically, at each time step, only a fraction of cells are
activated, and the activated cells select relevant inputs and cells to
communicate with. At the end of one time step, the hidden states of the
activated cells are updated by considering the relevance between the inputs and
the hidden states from the last and current time steps. Extensive experiments
on diversified sequential modeling tasks are conducted to show the superior
generalization ability when there exist changes in the testing environment.
Source code is available at https://github.com/ziyuwwang/rig-lstm.

通过利用 Grid LSTM 的模块化结构，在实际中的顺序过程中，仅激活一部分细胞并选择相关输入进行通信后，通过考虑输入、上一时刻和当前时刻的隐藏状态之间的关联性来更新隐藏状态，从而实现对测试环境变化的优越泛化能力。