Link prediction is a fundamental task in graph learning, inherently shaped by
the topology of the graph. While traditional heuristics are grounded in graph
topology, they encounter challenges in generalizing across diverse graphs.
Recent research efforts have aimed to leverage the potential of heuristics, yet
a unified formulation accommodating both local and global heuristics remains
undiscovered. Drawing insights from the fact that both local and global
heuristics can be represented by adjacency matrix multiplications, we propose a
unified matrix formulation to accommodate and generalize various heuristics. We
further propose the Heuristic Learning Graph Neural Network (HL-GNN) to
efficiently implement the formulation. HL-GNN adopts intra-layer propagation
and inter-layer connections, allowing it to reach a depth of around 20 layers
with lower time complexity than GCN. HL-GNN is proven to be more expressive
than heuristics and conventional GNNs, and it can adaptively trade-off between
node features and topological information. Extensive experiments on the
Planetoid, Amazon, and OGB datasets underscore the effectiveness and efficiency
of HL-GNN. It outperforms existing methods by a large margin in prediction
performance. Additionally, HL-GNN is several orders of magnitude faster than
heuristic-inspired methods while requiring only a few trainable parameters. The
case study further demonstrates that the generalized heuristics and learned
weights are highly interpretable.

利用统一矩阵公式和 HL-GNN 方法，本研究提出了一种整合了各类算启发式方法和传统图神经网络的学习模型，通过广泛的实验验证了其高效性和显著优于现有方法的预测性能。

基于图神经网络的启发式学习：一种用于链接预测的统一框架

Heuristic Learning with Graph Neural Networks: A Unified Framework for  Link Prediction

Support Vector Machines (SVM) have gathered significant acclaim as
classifiers due to their successful implementation of Statistical Learning
Theory. However, in the context of multiclass and multilabel settings, the
reliance on vector-based formulations in existing SVM-based models poses
limitations regarding flexibility and ease of incorporating additional terms to
handle specific challenges. To overcome these limitations, our research paper
focuses on introducing a matrix formulation for SVM that effectively addresses
these constraints. By employing the Accelerated Gradient Descent method in the
dual, we notably enhance the efficiency of solving the Matrix-SVM problem.
Experimental evaluations on multilabel and multiclass datasets demonstrate that
Matrix SVM achieves superior time efficacy while delivering similar results to
Binary Relevance SVM.
Moreover, our matrix formulation unveils crucial insights and advantages that
may not be readily apparent in traditional vector-based notations. We emphasize
that numerous multilabel models can be viewed as extensions of SVM, with
customised modifications to meet specific requirements. The matrix formulation
presented in this paper establishes a solid foundation for developing more
sophisticated models capable of effectively addressing the distinctive
challenges encountered in multilabel learning.

支持向量机 (SVM) 在统计学习理论的成功应用方面获得了显著的赞誉。然而，在多类和多标签设置环境中，现有基于向量表示的 SVM 模型的依赖性对于灵活性和易于整合附加项以处理特定挑战方面存在局限性。为了克服这些限制，我们的研究重点介绍了一个有效解决这些约束的 SVM 矩阵形式。通过在对偶问题中应用加速梯度下降方法，我们显著提高了解决矩阵 - SVM 问题的效率。对多标签和多类数据集的实验评估表明，矩阵 SVM 实现了更高的时间效率，并且与二元相关 SVM 的结果相似。此外，我们的矩阵形式揭示了在传统基于向量符号中可能不容易察觉的重要见解和优势。我们强调，许多多标签模型可以被视为 SVM 的扩展，通过定制修改以满足特定要求。本文提出的矩阵形式为开发更复杂的模型奠定了坚实的基础，能够有效应对多标签学习中遇到的独特挑战。

通过矩阵公式增强支持向量机中的模式分类

Enhancing Pattern Classification in Support Vector Machines through  Matrix Formulation

Large language models (LLMs) have brought significant and transformative
changes in human society. These models have demonstrated remarkable
capabilities in natural language understanding and generation, leading to
various advancements and impacts across several domains.
We consider the in-context learning under two formulation for attention
related regression in this work. Given matrices $A_1 \in \mathbb{R}^{n \times
d}$, and $A_2 \in \mathbb{R}^{n \times d}$ and $B \in \mathbb{R}^{n \times n}$,
the purpose is to solve some certain optimization problems: Normalized version
$\min_{X} \| D(X)^{-1} \exp(A_1 X A_2^\top) - B \|_F^2$ and Rescaled version
$\| \exp(A_1 X A_2^\top) - D(X) \cdot B \|_F^2$. Here $D(X) := \mathrm{diag}(
\exp(A_1 X A_2^\top) {\bf 1}_n )$.
Our regression problem shares similarities with previous studies on
softmax-related regression. Prior research has extensively investigated
regression techniques related to softmax regression: Normalized version $\|
\langle \exp(Ax) , {\bf 1}_n \rangle^{-1} \exp(Ax) - b \|_2^2$ and Resscaled
version $\| \exp(Ax) - \langle \exp(Ax), {\bf 1}_n \rangle b \|_2^2 $
In contrast to previous approaches, we adopt a vectorization technique to
address the regression problem in matrix formulation. This approach expands the
dimension from $d$ to $d^2$, resembling the formulation of the regression
problem mentioned earlier.
Upon completing the lipschitz analysis of our regression function, we have
derived our main result concerning in-context learning.

本文介绍了注意力相关回归问题在矩阵形式下的向量化技术及利普希茨分析结果。

基于张量技巧的上下文学习注意力机制：从单个 softmax 回归到多个 softmax 回归

In-Context Learning for Attention Scheme: from Single Softmax Regression  to Multiple Softmax Regression via a Tensor Trick

We consider methods for quantifying the similarity of vertices in networks.
We propose a measure of similarity based on the concept that two vertices are
similar if their immediate neighbors in the network are themselves similar.
This leads to a self-consistent matrix formulation of similarity that can be
evaluated iteratively using only a knowledge of the adjacency matrix of the
network. We test our similarity measure on computer-generated networks for
which the expected results are known, and on a number of real-world networks.

本文提出了一种基于顶点相邻度量相似性的方法，通过邻接矩阵迭代计算得到自洽矩阵，测试结果表明该方法可用于计算人工生成和真实网络中的节点相似性。