Despite recent successes in natural language processing and computer vision,
Transformer suffers from the scalability problem when dealing with graphs. The
computational complexity is unacceptable for large-scale graphs, e.g.,
knowledge graphs. One solution is to consider only the near neighbors, which,
however, will lose the key merit of Transformer to attend to the elements at
any distance. In this paper, we propose a new Transformer architecture, named
dual-encoding Transformer (DET). DET has a structural encoder to aggregate
information from connected neighbors and a semantic encoder to focus on
semantically useful distant nodes. In comparison with resorting to multi-hop
neighbors, DET seeks the desired distant neighbors via self-supervised
training. We further find these two encoders can be incorporated to boost each
others' performance. Our experiments demonstrate DET has achieved superior
performance compared to the respective state-of-the-art methods in dealing with
molecules, networks and knowledge graphs with various sizes.

本研究提出了一种名为 DET 的新型 Transformer 架构，采用结构编码器聚合连接邻居的信息和语义编码器聚焦于远距离节点，通过自监督训练寻找所需的远距离邻居，比使用多跳邻居的方法更为优越。实验结果表明，DET 在处理分子、网络和各种规模的知识图谱方面比对应的最新方法具有更优越的性能表现。

释放 Transformer 在图形任务中的能力

Unleashing the Power of Transformer for Graphs

RGBD (RGB plus depth) object tracking is gaining momentum as RGBD sensors
have become popular in many application fields such as robotics.However, the
best RGBD trackers are extensions of the state-of-the-art deep RGB trackers.
They are trained with RGB data and the depth channel is used as a sidekick for
subtleties such as occlusion detection. This can be explained by the fact that
there are no sufficiently large RGBD datasets to 1) train deep depth trackers
and to 2) challenge RGB trackers with sequences for which the depth cue is
essential. This work introduces a new RGBD tracking dataset - Depth-Track -
that has twice as many sequences (200) and scene types (40) than in the largest
existing dataset, and three times more objects (90). In addition, the average
length of the sequences (1473), the number of deformable objects (16) and the
number of annotated tracking attributes (15) have been increased. Furthermore,
by running the SotA RGB and RGBD trackers on DepthTrack, we propose a new RGBD
tracking baseline, namely DeT, which reveals that deep RGBD tracking indeed
benefits from genuine training data. The code and dataset is available at
this https URL

该论文介绍了一种新的 RGBD 跟踪数据集 - Depth-Track，其中有 200 个序列，40 种场景类型和 90 个对象，并建立了一个新的 RGBD 跟踪基准 DeT，揭示了真正的训练数据对深度 RGBD 跟踪的益处。