We build interpretable and lightweight transformer-like neural networks by
unrolling iterative optimization algorithms that minimize graph smoothness
priors -- the quadratic graph Laplacian regularizer (GLR) and the $\ell_1$-norm
graph total variation (GTV) -- subject to an interpolation constraint. The
crucial insight is that a normalized signal-dependent graph learning module
amounts to a variant of the basic self-attention mechanism in conventional
transformers. Unlike "black-box" transformers that require learning of large
key, query and value matrices to compute scaled dot products as affinities and
subsequent output embeddings, resulting in huge parameter sets, our unrolled
networks employ shallow CNNs to learn low-dimensional features per node to
establish pairwise Mahalanobis distances and construct sparse similarity
graphs. At each layer, given a learned graph, the target interpolated signal is
simply a low-pass filtered output derived from the minimization of an assumed
graph smoothness prior, leading to a dramatic reduction in parameter count.
Experiments for two image interpolation applications verify the restoration
performance, parameter efficiency and robustness to covariate shift of our
graph-based unrolled networks compared to conventional transformers.

通过迭代优化算法来构建可解释且轻量级的类 Transformer 神经网络，其中采用归一化的信号相关图学习模块代替传统 Transformer 中的基本自注意机制，通过学习每个节点的低维特征来建立配对的马哈拉诺比斯距离和构建稀疏相似度图，从而实现巨大参数集的减少和信号内插。与传统 Transformer 相比，图形未解卷积神经网络在图像内插应用中验证了恢复性能、参数效率和对协变量转换的鲁棒性。