BriefGPT.xyz
Oct, 2021
Transformer可推广至DeepSets,并可扩展至图和超图
Transformers Generalize DeepSets and Can be Extended to Graphs and Hypergraphs
HTML
PDF
Jinwoo Kim, Saeyoon Oh, Seunghoon Hong
TL;DR
本文将Transformer推广到任意排列不变数据(集合,图和超图),提出了高阶Transformer用于处理$k$阶数据,通过稀疏化和核化注意力等手段降低了复杂度并在大规模图回归和集合预测任务中获得了显著性能提升。
Abstract
We present a generalization of
transformers
to any-order
permutation invariant data
(sets, graphs, and hypergraphs). We begin by observing that
t
→