Time series modeling, a crucial area in science, often encounters challenges when training Machine Learning (ML) models like Recurrent Neural Networks (RNNs) using the conventional mini-batch training strategy that assumes independent and identically distributed (IID) samples and initializes RNNs with zero hidden states. The IID assumption ignores temporal dependencies among samples, resulting in poor performance. This paper proposes the Message Propagation Through Time (MPTT) algorithm to effectively incorporate long temporal dependencies while preserving faster training times relative to the stateful solutions. MPTT utilizes two memory modules to asynchronously manage initial hidden states for RNNs, fostering seamless information exchange between samples and allowing diverse mini-batches throughout epochs. MPTT further implements three policies to filter outdated and preserve essential information in the hidden states to generate informative initial hidden states for RNNs, facilitating robust training. Experimental results demonstrate that MPTT outperforms seven strategies on four climate datasets with varying levels of temporal dependencies.

该论文介绍了通过时间传播信息的算法（MPTT），它有效地结合了长期的时间依赖性，并相对于有状态的解决方案保持更快的训练时间。该算法利用两个内存模块异步管理RNN的初始隐藏状态，促进样本之间的无缝信息交流，并允许在整个周期中使用不同的小批量样本。MPTT还实施了三个策略来过滤过时的信息，并保留隐藏状态中的关键信息，以生成信息丰富的初始隐藏状态，从而实现鲁棒训练。实验结果表明，MPTT在具有不同级别时间依赖性的四个气候数据集上优于七种策略。

信息时序中的消息传播：一种序列依赖保留的算法