This paper focuses on learning representation on the whole graph level in an
unsupervised manner. Learning graph-level representation plays an important
role in a variety of real-world issues such as molecule property prediction,
protein structure feature extraction, and social network analysis. The
mainstream method is utilizing contrastive learning to facilitate graph feature
extraction, known as Graph Contrastive Learning (GCL). GCL, although effective,
suffers from some complications in contrastive learning, such as the effect of
false negative pairs. Moreover, augmentation strategies in GCL are weakly
adaptive to diverse graph datasets. Motivated by these problems, we propose a
novel framework called Structure Knowledge Refinement (SKR) which uses data
structure to determine the probability of whether a pair is positive or
negative. Meanwhile, we propose an augmentation strategy that naturally
preserves the semantic meaning of the original data and is compatible with our
SKR framework. Furthermore, we illustrate the effectiveness of our SKR
framework through intuition and experiments. The experimental results on the
tasks of graph-level classification demonstrate that our SKR framework is
superior to most state-of-the-art baselines.

本文提出了一种名为 “结构知识细化” 的新框架，用于在无监督的情况下学习整个图形级别的表示，该框架通过数据结构确定一对数据是否为正或负，并通过直观观察和实验证明了其在图形级别分类任务上优于大多数最先进的对照方法。

基于结构知识精细化的图级蛋白质表示学习

Graph-level Protein Representation Learning by Structure Knowledge  Refinement

Predicting program properties such as names or expression types has a wide
range of applications. It can ease the task of programming and increase
programmer productivity. A major challenge when learning from programs is
$\textit{how to represent programs in a way that facilitates effective
learning}$.
We present a $\textit{general path-based representation}$ for learning from
programs. Our representation is purely syntactic and extracted automatically.
The main idea is to represent a program using paths in its abstract syntax tree
(AST). This allows a learning model to leverage the structured nature of code
rather than treating it as a flat sequence of tokens.
We show that this representation is general and can: (i) cover different
prediction tasks, (ii) drive different learning algorithms (for both generative
and discriminative models), and (iii) work across different programming
languages.
We evaluate our approach on the tasks of predicting variable names, method
names, and full types. We use our representation to drive both CRF-based and
word2vec-based learning, for programs of four languages: JavaScript, Java,
Python and C\#. Our evaluation shows that our approach obtains better results
than task-specific handcrafted representations across different tasks and
programming languages.

提出一种基于路径的编程语言自动学习表示方法，其将程序表示为其抽象语法树中的路径，可用于预测变量名称、方法名称和完全类型，并可跨不同编程语言驱动不同的学习算法。

基于路径的通用程序属性预测表示

A General Path-Based Representation for Predicting Program Properties

How much can pruning algorithms teach us about the fundamentals of learning
representations in neural networks? And how much can these fundamentals help
while devising new pruning techniques? A lot, it turns out. Neural network
pruning has become a topic of great interest in recent years, and many
different techniques have been proposed to address this problem. The decision
of what to prune and when to prune necessarily forces us to confront our
assumptions about how neural networks actually learn to represent patterns in
data. In this work, we set out to test several long-held hypotheses about
neural network learning representations, approaches to pruning and the
relevance of one in the context of the other. To accomplish this, we argue in
favor of pruning whole neurons as opposed to the traditional method of pruning
weights from optimally trained networks. We first review the historical
literature, point out some common assumptions it makes, and propose methods to
demonstrate the inherent flaws in these assumptions. We then propose our novel
approach to pruning and set about analyzing the quality of the decisions it
makes. Our analysis led us to question the validity of many widely-held
assumptions behind pruning algorithms and the trade-offs we often make in the
interest of reducing computational complexity. We discovered that there is a
straightforward way, however expensive, to serially prune 40-70% of the neurons
in a trained network with minimal effect on the learning representation and
without any re-training. It is to be noted here that the motivation behind this
work is not to propose an algorithm that would outperform all existing methods,
but to shed light on what some inherent flaws in these methods can teach us
about learning representations and how this can lead us to superior pruning
techniques.

本文探讨神经网络剪枝问题，在对历史文献进行回顾及常见假设进行分析后，提出了一种新型的神经元全剪枝方法，得出存在许多基于剪枝算法的固有缺陷及为减少计算复杂性而做出的权衡。另外，还发现剪去 40-70% 的神经元实际上对学习表示形式并没有太大的影响。