We investigate the role of attention and memory in complex reasoning tasks.
We analyze Transformer-based self-attention as a model and extend it with
memory. By studying a synthetic visual reasoning test, we refine the taxonomy
of reasoning tasks. Incorporating self-attention with ResNet50, we enhance
feature maps using feature-based and spatial attention, achieving efficient
solving of challenging visual reasoning tasks. Our findings contribute to
understanding the attentional needs of SVRT tasks. Additionally, we propose
GAMR, a cognitive architecture combining attention and memory, inspired by
active vision theory. GAMR outperforms other architectures in sample
efficiency, robustness, and compositionality, and shows zero-shot
generalization on new reasoning tasks.

本文研究了复杂的推理任务中注意力和记忆的作用，分析了 Transformer-based self-attention 模型，并通过增加记忆单元的方式对其进行了扩展，提出了一种结合了注意力和记忆的认知架构 GAMR，并在视觉推理测试中的表现超过了其他架构。

探究 (自我) 关注机制在认知与计算机视觉架构中的作用的博士论文

PhD Thesis: Exploring the role of (self-)attention in cognitive and  computer vision architecture

Deep imitation learning is a promising approach that does not require
hard-coded control rules in autonomous robot manipulation. The current
applications of deep imitation learning to robot manipulation have been limited
to reactive control based on the states at the current time step. However,
future robots will also be required to solve tasks utilizing their memory
obtained by experience in complicated environments (e.g., when the robot is
asked to find a previously used object on a shelf). In such a situation, simple
deep imitation learning may fail because of distractions caused by complicated
environments. We propose that gaze prediction from sequential visual input
enables the robot to perform a manipulation task that requires memory. The
proposed algorithm uses a Transformer-based self-attention architecture for the
gaze estimation based on sequential data to implement memory. The proposed
method was evaluated with a real robot multi-object manipulation task that
requires memory of the previous states.

本文提出了一种使用基于 Transformer 的自注意力结构实现眼神预测，以实现具有记忆的机器人操作任务的算法。通过将机器人视觉输入的连续序列作为输入，该算法能够通过使用传统的深度模仿学习方法并结合眼动预测来实现多物体操作任务。实验结果表明，该方法成功地改善了机器人在具有记忆要求的复杂环境中的操作表现。

机器人操作中基于深度模仿学习的记忆驱动注视预测

Memory-based gaze prediction in deep imitation learning for robot manipulation

Graph neural networks are powerful architectures for structured datasets.
However, current methods struggle to represent long-range dependencies. Scaling
the depth or width of GNNs is insufficient to broaden receptive fields as
larger GNNs encounter optimization instabilities such as vanishing gradients
and representation oversmoothing, while pooling-based approaches have yet to
become as universally useful as in computer vision. In this work, we propose
the use of Transformer-based self-attention to learn long-range pairwise
relationships, with a novel "readout" mechanism to obtain a global graph
embedding. Inspired by recent computer vision results that find
position-invariant attention performant in learning long-range relationships,
our method, which we call GraphTrans, applies a permutation-invariant
Transformer module after a standard GNN module. This simple architecture leads
to state-of-the-art results on several graph classification tasks,
outperforming methods that explicitly encode graph structure. Our results
suggest that purely-learning-based approaches without graph structure may be
suitable for learning high-level, long-range relationships on graphs. Code for
GraphTrans is available at this https URL

本篇论文提出了一种名为 GraphTrans 的算法，它将 Transformer 自注意力模块和标准的 GNN 模块相结合，并应用置换不变性的 Transformer 模块，从而解决了当前方法在表示长程依赖关系方面的不足。该算法在多个图分类任务上表现出色，说明纯粹的基于学习的方法可能适用于在图上学习高水平、长程的关系。