We introduce eigenpruning, a method that removes singular values from weight
matrices in an LLM to improve its performance in a particular task. This method
is inspired by interpretability methods designed to automatically find
subnetworks of a model which solve a specific task. In our tests, the pruned
model outperforms the original model by a large margin, while only requiring
minimal computation to prune the weight matrices. In the case of a small
synthetic task in integer multiplication, the Phi-2 model can improve its
accuracy in the test set from 13.75% to 97.50%. Interestingly, these results
seem to indicate the existence of a computation path that can solve the task
very effectively, but it was not being used by the original model. Finally, we
plan to open-source our implementation in the camera-ready version of our work.

我们介绍了一种叫做 eigenpruning 的方法，它从 LLM 中移除特定任务中的奇异值，以提高其性能。这种方法受到解释性方法的启发，旨在自动找到解决特定任务的模型的子网络。在我们的测试中，经过修剪的模型在性能上远远优于原始模型，而仅需进行最少的计算来修剪权重矩阵。对于一个小的合成任务，即整数乘法，Phi-2 模型可以将测试集的准确率从 13.75％提高到 97.50％。有趣的是，这些结果似乎表明存在一种非常有效地解决任务的计算路径，但原始模型未使用。最后，我们计划在我们的工作的最终版本中开源我们的实现。

特征修剪

Eigenpruning

Table reasoning, which aims to generate the corresponding answer to the
question following the user requirement according to the provided table, and
optionally a text description of the table, effectively improving the
efficiency of obtaining information. Recently, using Large Language Models
(LLMs) has become the mainstream method for table reasoning, because it not
only significantly reduces the annotation cost but also exceeds the performance
of previous methods. However, existing research still lacks a summary of
LLM-based table reasoning works. Due to the existing lack of research,
questions about which techniques can improve table reasoning performance in the
era of LLMs, why LLMs excel at table reasoning, and how to enhance table
reasoning abilities in the future, remain largely unexplored. This gap
significantly limits progress in research. To answer the above questions and
advance table reasoning research with LLMs, we present this survey to analyze
existing research, inspiring future work. In this paper, we analyze the
mainstream techniques used to improve table reasoning performance in the LLM
era, and the advantages of LLMs compared to pre-LLMs for solving table
reasoning. We provide research directions from both the improvement of existing
methods and the expansion of practical applications to inspire future research.

使用大型语言模型 (LLMs) 改善表格推理性能是当前主流方法。本文分析了在 LLMs 时代中提升表格推理性能的主要技术以及 LLMs 相较于之前方法的优势，并提供了改进现有方法和扩展实际应用的研究方向，以激发未来表格推理研究的进一步发展。

大型语言模型在表格推理中的调查

A Survey of Table Reasoning with Large Language Models

Explainable Artificial Intelligence (XAI) aims to provide insights into the
decision-making process of AI models, allowing users to understand their
results beyond their decisions. A significant goal of XAI is to improve the
performance of AI models by providing explanations for their decision-making
processes. However, most XAI literature focuses on how to explain an AI system,
while less attention has been given to how XAI methods can be exploited to
improve an AI system. In this work, a set of well-known XAI methods typically
used with Machine Learning (ML) classification tasks are investigated to verify
if they can be exploited, not just to provide explanations but also to improve
the performance of the model itself. To this aim, two strategies to use the
explanation to improve a classification system are reported and empirically
evaluated on three datasets: Fashion-MNIST, CIFAR10, and STL10. Results suggest
that explanations built by Integrated Gradients highlight input features that
can be effectively used to improve classification performance.

本研究探讨了一组通常与机器学习分类任务一起使用的著名可解释人工智能 (XAI) 方法，以验证它们是否能被利用，不仅提供解释，而且还可以改善模型本身的性能。研究结果表明，Integrated Gradients 方法构建的解释突出了可用于提高分类性能的输入特征。三个数据集上的实证评估支持了这一结论。