We explore a new class of brain encoding model by adding memory-related
information as input. Memory is an essential brain mechanism that works
alongside visual stimuli. During a vision-memory cognitive task, we found the
non-visual brain is largely predictable using previously seen images. Our
Memory Encoding Model (Mem) won the Algonauts 2023 visual brain competition
even without model ensemble (single model score 66.8, ensemble score 70.8). Our
ensemble model without memory input (61.4) can also stand a 3rd place.
Furthermore, we observe periodic delayed brain response correlated to 6th-7th
prior image, and hippocampus also showed correlated activity timed with this
periodicity. We conjuncture that the periodic replay could be related to memory
mechanism to enhance the working memory.

我们通过将与记忆相关的信息作为输入来探索一种新的脑编码模型。在视觉 - 记忆认知任务中，我们发现非视觉脑部在很大程度上可以通过先前看到的图像进行准确预测。我们的记忆编码模型（Mem）在 Algonauts 2023 视觉脑竞赛中获胜，即使没有模型集成（单一模型得分 66.8，集成得分 70.8）。我们的没有记忆输入的集成模型（61.4）也获得了第三名。此外，我们观察到周期性延迟的脑响应与第 6-7 个先前图像相关，并且海马体的活动也与这种周期性保持一致。我们推测周期性重放可能与增强工作记忆的记忆机制有关。