Model scaling is becoming the default choice for many language tasks due to
the success of large language models (LLMs). However, it can fall short in
specific scenarios where simple customized methods excel. In this paper, we
delve into the patent approval pre-diction task and unveil that simple
domain-specific graph methods outperform enlarging the model, using the
intrinsic dependencies within the patent data. Specifically, we first extend
the embedding-based state-of-the-art (SOTA) by scaling up its backbone model
with various sizes of open-source LLMs, then explore prompt-based methods to
harness proprietary LLMs' potential, but find the best results close to random
guessing, underlining the ineffectiveness of model scaling-up. Hence, we
propose a novel Fine-grained cLAim depeNdency (FLAN) Graph through meticulous
patent data analyses, capturing the inherent dependencies across segments of
the patent text. As it is model-agnostic, we apply cost-effective graph models
to our FLAN Graph to obtain representations for approval prediction. Extensive
experiments and detailed analyses prove that incorporating FLAN Graph via
various graph models consistently outperforms all LLM baselines significantly.
We hope that our observations and analyses in this paper can bring more
attention to this challenging task and prompt further research into the
limitations of LLMs. Our source code and dataset can be obtained from
this http URL

通过采用简单的专业领域图方法，我们揭示了在专利批准预测任务中，模型扩展可能存在不足之处，并证明了显式捕获专利文本的内在依赖关系的图表示方法通过各种图模型的应用，始终明显优于所有大型语言模型基线。

超越规模：利用领域特定的细粒度权利要求依赖图预测专利批准

Beyond Scaling: Predicting Patent Approval with Domain-specific  Fine-grained Claim Dependency Graph

Given the success of Graph Neural Networks (GNNs) for structure-aware machine
learning, numerous studies have explored their application to text
classification, as an alternative to traditional feature representation models.
However, most studies considered just a specific domain and validated on data
with particular characteristics. This work presents an extensive empirical
investigation of graph-based text representation methods proposed for text
classification, identifying practical implications and open challenges in the
field. We compare several GNN architectures as well as BERT across five
datasets, encompassing short and also long documents. The results show that: i)
graph performance is highly related to the textual input features and domain,
ii) despite its outstanding performance, BERT has difficulties converging when
dealing with short texts, iii) graph methods are particularly beneficial for
longer documents.

本研究对基于图的文本表示方法在文本分类中的应用进行了广泛的实证研究，探索了其实际影响和开放挑战。在短文档和长文档数据集上比较了几个 GNN 架构和 BERT 的性能，并发现文本输入特征和领域高度相关，尽管 BERT 表现出色，但在处理短文本时存在收敛困难，而图方法特别有利于处理较长文本。

如何使用基于图神经网络的最佳文本表示法进行文本分类：连接数据点？

Connecting the Dots: What Graph-Based Text Representations Work Best for  Text Classification using Graph Neural Networks?

Anti-money laundering (AML) regulations play a critical role in safeguarding
financial systems, but bear high costs for institutions and drive financial
exclusion for those on the socioeconomic and international margins. The advent
of cryptocurrency has introduced an intriguing paradox: pseudonymity allows
criminals to hide in plain sight, but open data gives more power to
investigators and enables the crowdsourcing of forensic analysis. Meanwhile
advances in learning algorithms show great promise for the AML toolkit. In this
workshop tutorial, we motivate the opportunity to reconcile the cause of safety
with that of financial inclusion. We contribute the Elliptic Data Set, a time
series graph of over 200K Bitcoin transactions (nodes), 234K directed payment
flows (edges), and 166 node features, including ones based on non-public data;
to our knowledge, this is the largest labelled transaction data set publicly
available in any cryptocurrency. We share results from a binary classification
task predicting illicit transactions using variations of Logistic Regression
(LR), Random Forest (RF), Multilayer Perceptrons (MLP), and Graph Convolutional
Networks (GCN), with GCN being of special interest as an emergent new method
for capturing relational information. The results show the superiority of
Random Forest (RF), but also invite algorithmic work to combine the respective
powers of RF and graph methods. Lastly, we consider visualization for analysis
and explainability, which is difficult given the size and dynamism of
real-world transaction graphs, and we offer a simple prototype capable of
navigating the graph and observing model performance on illicit activity over
time. With this tutorial and data set, we hope to a) invite feedback in support
of our ongoing inquiry, and b) inspire others to work on this societally
important challenge.

本文介绍了反洗钱规定在维护金融系统安全方面扮演的关键角色，以及加密货币所引入的假名化和开放数据等新问题，同时提出使用分类算法和图方法来预测和控制非法交易，并分享了一个被标记的比特币交易数据集以供进一步研究。