In both the human brain and any general artificial intelligence (AI), a
representation of the past is necessary to predict the future. However, perfect
storage of all experiences is not feasible. One approach utilized in many
applications, including reward prediction in reinforcement learning, is to
retain recently active features of experience in a buffer. Despite its prior
successes, we show that the fixed length buffer renders Deep Q-learning
Networks (DQNs) fragile to changes in the scale over which information can be
learned. To enable learning when the relevant temporal scales in the
environment are not known *a priori*, recent advances in psychology and
neuroscience suggest that the brain maintains a compressed representation of
the past. Here we introduce a neurally-plausible, scale-free memory
representation we call Scale-Invariant Temporal History (SITH) for use with
artificial agents. This representation covers an exponentially large period of
time by sacrificing temporal accuracy for events further in the past. We
demonstrate the utility of this representation by comparing the performance of
agents given SITH, buffer, and exponential decay representations in learning to
play video games at different levels of complexity. In these environments, SITH
exhibits better learning performance by storing information for longer
timescales than a fixed-size buffer, and representing this information more
clearly than a set of exponentially decayed features. Finally, we discuss how
the application of SITH, along with other human-inspired models of cognition,
could improve reinforcement and machine learning algorithms in general.

介绍了一种用于人工智能应用的神经可行的、免于尺度束缚的记忆表示方法 ——SITH，SITH 在学习复杂任务时能够比固定大小的缓存和指数衰减特征存储更好地存储信息，表现更好。

尺度不变的时间历史（SITH）：在不确定的世界中过去的最佳切片

Scale-invariant temporal history (SITH): optimal slicing of the past in  an uncertain world

We introduce a new test of how well language models capture meaning in
children's books. Unlike standard language modelling benchmarks, it
distinguishes the task of predicting syntactic function words from that of
predicting lower-frequency words, which carry greater semantic content. We
compare a range of state-of-the-art models, each with a different way of
encoding what has been previously read. We show that models which store
explicit representations of long-term contexts outperform state-of-the-art
neural language models at predicting semantic content words, although this
advantage is not observed for syntactic function words. Interestingly, we find
that the amount of text encoded in a single memory representation is highly
influential to the performance: there is a sweet-spot, not too big and not too
small, between single words and full sentences that allows the most meaningful
information in a text to be effectively retained and recalled. Further, the
attention over such window-based memories can be trained effectively through
self-supervision. We then assess the generality of this principle by applying
it to the CNN QA benchmark, which involves identifying named entities in
paraphrased summaries of news articles, and achieve state-of-the-art
performance.

介绍了一种新的测试语言模型在儿童书籍中捕捉意义的方法，通过与一系列现有的模型进行比较，发现长期语境的显式表示可以比神经语言模型更好地预测语义内容的单词，并且单一记忆表示中编码的文本数量非常有利于性能，同时还可通过自我监督有效地训练对其进行注意。同时，将这一原则应用于 CNN QA 基准测试中时，也取得了最好的结果。