This paper presents a novel approach to address the challenge of online
hidden representation learning for decision-making under uncertainty in
non-stationary, partially observable environments. The proposed algorithm,
Distributed Hebbian Temporal Memory (DHTM), is based on factor graph formalism
and a multicomponent neuron model. DHTM aims to capture sequential data
relationships and make cumulative predictions about future observations,
forming Successor Representation (SR). Inspired by neurophysiological models of
the neocortex, the algorithm utilizes distributed representations, sparse
transition matrices, and local Hebbian-like learning rules to overcome the
instability and slow learning process of traditional temporal memory algorithms
like RNN and HMM. Experimental results demonstrate that DHTM outperforms
classical LSTM and performs comparably to more advanced RNN-like algorithms,
speeding up Temporal Difference learning for SR in changing environments.
Additionally, we compare the SRs produced by DHTM to another biologically
inspired HMM-like algorithm, CSCG. Our findings suggest that DHTM is a
promising approach for addressing the challenges of online hidden
representation learning in dynamic environments.

这篇论文介绍了一种针对不稳定、部分可观测环境下决策的在线隐藏表示学习方法。提出的分布式赫比时态记忆（DHTM）算法基于因子图形式和多组分神经元模型，它旨在捕捉序列数据关系，并对未来观察作出累积预测，形成继承者表示。经实验证明，DHTM 算法在应对动态环境中的时序差异学习方面表现优于经典 LSTM，并与更高级的类似 RNN 的算法相当，加速了继承者表示中的时间差异学习。此外，我们将 DHTM 产生的继承者表示与另一种生物启发的类 HMM 算法 CSCG 进行了比较，发现 DHTM 是解决在线隐藏表示学习动态环境挑战的一种有前途的方法。