Effectively learning from sequential data is a longstanding goal of
Artificial Intelligence, especially in the case of long sequences. From the
dawn of Machine Learning, several researchers engaged in the search of
algorithms and architectures capable of processing sequences of patterns,
retaining information about the past inputs while still leveraging the upcoming
data, without losing precious long-term dependencies and correlations. While
such an ultimate goal is inspired by the human hallmark of continuous real-time
processing of sensory information, several solutions simplified the learning
paradigm by artificially limiting the processed context or dealing with
sequences of limited length, given in advance. These solutions were further
emphasized by the large ubiquity of Transformers, that have initially shaded
the role of Recurrent Neural Nets. However, recurrent networks are facing a
strong recent revival due to the growing popularity of (deep) State-Space
models and novel instances of large-context Transformers, which are both based
on recurrent computations to go beyond several limits of currently ubiquitous
technologies. In fact, the fast development of Large Language Models enhanced
the interest in efficient solutions to process data over time. This survey
provides an in-depth summary of the latest approaches that are based on
recurrent models for sequential data processing. A complete taxonomy over the
latest trends in architectural and algorithmic solutions is reported and
discussed, guiding researchers in this appealing research field. The emerging
picture suggests that there is room for thinking of novel routes, constituted
by learning algorithms which depart from the standard Backpropagation Through
Time, towards a more realistic scenario where patterns are effectively
processed online, leveraging local-forward computations, opening to further
research on this topic.

对基于循环模型的顺序数据处理的最新方法进行了深入总结，并提供了关于体系结构和算法解决方案的完整分类，引导研究者在这一吸引人的研究领域进行进一步研究。