Multi-modal integration and classification based on graph learning is among
the most challenging obstacles in disease prediction due to its complexity.
Several recent works on the basis of attentional mechanisms have been proposed
to disentangle the problem of multi-modal integration. However, there are
certain limitations to these techniques. Primarily, these works focus on
explicitly integrating at the feature level using weight scores, which cannot
effectively address the negative impact between modalities. Next, a majority of
them utilize single-sized filters to extract graph features, ignoring the
heterogeneous information over graphs. To overcome these drawbacks, we propose
MMKGL (Multi-modal Multi-Kernel Graph Learning). For the problem of negative
impact between modalities, we use the multi-modal graph embedding module to
construct a multi-modal graph. Different from the traditional manual
construction of static graphs, a separate graph is generated for each modality
by graph adaptive learning, where a function graph and a supervision graph are
introduced for optimiztion during the multi-graph fusion embedding process. We
then apply the multi-kernel graph learning module to extract heterogeneous
information from the multi-modal graph. The information in the multi-modal
graph at different levels is aggregated by convolutional kernels with different
receptive field sizes, followed by generating a cross-kernel discovery tensor
for disease prediction. Our method is evaluated on the benchmark Autism Brain
Imaging Data Exchange (ABIDE) dataset and outperforms the state-of-the-art
methods. In addition, discriminative brain regions associated with autism are
identified by our model, providing guidance for the study of autism pathology.

通过提出 MMKGL（Multi-modal Multi-Kernel Graph Learning）来解决多模态图学习的问题，利用该方法提取多模态图中的异质信息，基于 Autism Brain Imaging Data Exchange (ABIDE) 数据集进行评估并识别与自闭症相关的脑区。

自闭症预测和生物标志物发现的多模态多核图学习

Multi-modal Multi-kernel Graph Learning for Autism Prediction and Biomarker Discovery

Attentional mechanisms are order-invariant. Positional encoding is a crucial
component to allow attention-based deep model architectures such as Transformer
to address sequences or images where the position of information matters. In
this paper, we propose a novel positional encoding method based on learnable
Fourier features. Instead of hard-coding each position as a token or a vector,
we represent each position, which can be multi-dimensional, as a trainable
encoding based on learnable Fourier feature mapping, modulated with a
multi-layer perceptron. The representation is particularly advantageous for a
spatial multi-dimensional position, e.g., pixel positions on an image, where
$L_2$ distances or more complex positional relationships need to be captured.
Our experiments based on several public benchmark tasks show that our learnable
Fourier feature representation for multi-dimensional positional encoding
outperforms existing methods by both improving the accuracy and allowing faster
convergence.

本文提出了一种基于可学习的 Fourier 特征的新型位置编码方法，能够优化空间多维位置的表示、提高精度和加速收敛速度。

可学习的傅里叶特征用于多维空间位置编码

Learnable Fourier Features for Multi-Dimensional Spatial Positional  Encoding

Attention has long been proposed by psychologists as important for
effectively dealing with the enormous sensory stimulus available in the
neocortex. Inspired by the visual attention models in computational
neuroscience and the need of object-centric data for generative models, we
describe for generative learning framework using attentional mechanisms.
Attentional mechanisms can propagate signals from region of interest in a scene
to an aligned canonical representation, where generative modeling takes place.
By ignoring background clutter, generative models can concentrate their
resources on the object of interest. Our model is a proper graphical model
where the 2D Similarity transformation is a part of the top-down process. A
ConvNet is employed to provide good initializations during posterior inference
which is based on Hamiltonian Monte Carlo. Upon learning images of faces, our
model can robustly attend to face regions of novel test subjects. More
importantly, our model can learn generative models of new faces from a novel
dataset of large images where the face locations are not known.

通过利用关注机制的生成学习框架，可以将图像中感兴趣的区域传递给生成建模的标准表征，并集中精力在感兴趣的对象上，从而建立新面孔的生成模型。