Pluggable type systems allow programmers to extend the type system of a
programming language to enforce semantic properties defined by the programmer.
Pluggable type systems are difficult to deploy in legacy codebases because they
require programmers to write type annotations manually. This paper investigates
how to use machine learning to infer type qualifiers automatically. We propose
a novel representation, NaP-AST, that encodes minimal dataflow hints for the
effective inference of type qualifiers. We evaluate several model architectures
for inferring type qualifiers, including Graph Transformer Network, Graph
Convolutional Network and Large Language Model. We further validated these
models by applying them to 12 open-source programs from a prior evaluation of
the NullAway pluggable typechecker, lowering warnings in all but one
unannotated project. We discovered that GTN shows the best performance, with a
recall of .89 and precision of 0.6. Furthermore, we conduct a study to estimate
the number of Java classes needed for good performance of the trained model.
For our feasibility study, performance improved around 16k classes, and
deteriorated due to overfitting around 22k classes.

使用机器学习自动推断类型限定符的可插入式类型系统研究，提出了一种编码最小数据流提示的新型表示方式 NaP-AST，并评估了多种模型架构，包括图变换网络（Graph Transformer Network），图卷积网络（Graph Convolutional Network）和大型语言模型。研究中还验证了这些模型在 12 个开源项目中的性能，并进行了一个可行性研究，结果发现在大约 16k 个类时性能有所提升，在约 22k 个类时由于过拟合而恶化。

用机器学习推断可插拔类型

Inferring Pluggable Types with Machine Learning

Machine reading comprehension has aroused wide concerns, since it explores
the potential of model for text understanding. To further equip the machine
with the reasoning capability, the challenging task of logical reasoning is
proposed. Previous works on logical reasoning have proposed some strategies to
extract the logical units from different aspects. However, there still remains
a challenge to model the long distance dependency among the logical units.
Also, it is demanding to uncover the logical structures of the text and further
fuse the discrete logic to the continuous text embedding. To tackle the above
issues, we propose an end-to-end model Logiformer which utilizes a two-branch
graph transformer network for logical reasoning of text. Firstly, we introduce
different extraction strategies to split the text into two sets of logical
units, and construct the logical graph and the syntax graph respectively. The
logical graph models the causal relations for the logical branch while the
syntax graph captures the co-occurrence relations for the syntax branch.
Secondly, to model the long distance dependency, the node sequence from each
graph is fed into the fully connected graph transformer structures. The two
adjacent matrices are viewed as the attention biases for the graph transformer
layers, which map the discrete logical structures to the continuous text
embedding space. Thirdly, a dynamic gate mechanism and a question-aware
self-attention module are introduced before the answer prediction to update the
features. The reasoning process provides the interpretability by employing the
logical units, which are consistent with human cognition. The experimental
results show the superiority of our model, which outperforms the
state-of-the-art single model on two logical reasoning benchmarks.

Logiformer 是采用图形变换器网络的端到端模型，具有解决逻辑推理中长距离依赖性问题和将离散逻辑结构映射到连续文本嵌入空间的优点，介绍了不同的逻辑单位抽取策略，采用动态门机制和问题感知的自我注意模块在答案预测之前更新特征，其推理过程利用逻辑单位提供了可解释性，实验证明其优于现有最先进的单一模型在两个逻辑推理基准测试上的性能。

Logiformer：一种用于解释性逻辑推理的双分支图转换器网络

Logiformer: A Two-Branch Graph Transformer Network for Interpretable Logical Reasoning

Accurate user and item embedding learning is crucial for modern recommender
systems. However, most existing recommendation techniques have thus far focused
on modeling users' preferences over singular type of user-item interactions.
Many practical recommendation scenarios involve multi-typed user interactive
behaviors (e.g., page view, add-to-favorite and purchase), which presents
unique challenges that cannot be handled by current recommendation solutions.
In particular: i) complex inter-dependencies across different types of user
behaviors; ii) the incorporation of knowledge-aware item relations into the
multi-behavior recommendation framework; iii) dynamic characteristics of
multi-typed user-item interactions. To tackle these challenges, this work
proposes a Knowledge-Enhanced Hierarchical Graph Transformer Network (KHGT), to
investigate multi-typed interactive patterns between users and items in
recommender systems. Specifically, KHGT is built upon a graph-structured neural
architecture to i) capture type-specific behavior characteristics; ii)
explicitly discriminate which types of user-item interactions are more
important in assisting the forecasting task on the target behavior.
Additionally, we further integrate the graph attention layer with the temporal
encoding strategy, to empower the learned embeddings be reflective of both
dedicated multiplex user-item and item-item relations, as well as the
underlying interaction dynamics. Extensive experiments conducted on three
real-world datasets show that KHGT consistently outperforms many
state-of-the-art recommendation methods across various evaluation settings. Our
implementation code is available at this https URL

本文提出了一种知识增强的分层图转换网络，以调查推荐系统中用户和物品之间的多类型互动模式，包括复杂的用户行为特征，知识感知的物品关系以及多类型用户物品交互的动态特性，实验表明该方法在多种情况下优于现有的推荐方法。