Table reasoning has shown remarkable progress in a wide range of table-based
tasks. These challenging tasks require reasoning over both free-form natural
language (NL) questions and semi-structured tabular data. However, previous
table reasoning solutions suffer from significant performance degradation on
"huge" tables. In addition, most existing methods struggle to reason over
complex questions since they lack essential information or they are scattered
in different places. To alleviate these challenges, we exploit a table
provider, namely TAP4LLM, on versatile sampling, augmentation, and packing
methods to achieve effective semi-structured data reasoning using large
language models (LLMs), which 1) decompose raw tables into sub-tables with
specific rows or columns based on the rules or semantic similarity; 2) augment
table information by extracting semantic and statistical metadata from raw
tables while retrieving relevant knowledge from trustworthy knowledge sources
(e.g., Wolfram Alpha, Wikipedia); 3) pack sampled tables with augmented
knowledge into sequence prompts for LLMs reasoning while balancing the token
allocation trade-off. We show that TAP4LLM allows for different components as
plug-ins, enhancing LLMs' understanding of structured data in diverse tabular
tasks.

通过使用大型语言模型（LLMs）和 TAP4LLM 表格提供程序，我们在各种表格任务中实现了有效的半结构化数据推理，包括将原始表格分解为子表、提取语义和统计元数据、从可信知识源检索相关知识以及为 LLMs 推理打包采样表格。TAP4LLM 的不同组件作为插件，增强了 LLMs 对多样化表格任务中结构化数据的理解。

TAP4LLM：针对大型语言模型推理的半结构化数据采样、增补和打包表格提供器

TAP4LLM: Table Provider on Sampling, Augmenting, and Packing  Semi-structured Data for Large Language Model Reasoning

Recent literature has suggested the potential of using large language models
(LLMs) to make predictions for tabular tasks. However, LLMs have been shown to
exhibit harmful social biases that reflect the stereotypes and inequalities
present in the society. To this end, as well as the widespread use of tabular
data in many high-stake applications, it is imperative to explore the following
questions: what sources of information do LLMs draw upon when making
predictions for tabular tasks; whether and to what extent are LLM predictions
for tabular tasks influenced by social biases and stereotypes; and what are the
consequential implications for fairness? Through a series of experiments, we
delve into these questions and show that LLMs tend to inherit social biases
from their training data which significantly impact their fairness in tabular
prediction tasks. Furthermore, our investigations show that in the context of
bias mitigation, though in-context learning and fine-tuning have a moderate
effect, the fairness metric gap between different subgroups is still larger
than that in traditional machine learning models, such as Random Forest and
shallow Neural Networks. This observation emphasizes that the social biases are
inherent within the LLMs themselves and inherited from their pre-training
corpus, not only from the downstream task datasets. Besides, we demonstrate
that label-flipping of in-context examples can significantly reduce biases,
further highlighting the presence of inherent bias within LLMs.

通过一系列的实验，我们发现大型语言模型在表格预测任务中往往会继承社会偏见，这从根源上影响了它们的公平性，并且通过标签反转等方法可以显著减少这些偏见。

对大型语言模型在表格数据预测中的公平性调查

Investigating the Fairness of Large Language Models for Predictions on  Tabular Data

Large language models (LLMs) are increasingly applied for tabular tasks using
in-context learning. The prompt representation for a table may play a role in
the LLMs ability to process the table. Inspired by prior work, we generate a
collection of self-supervised structural tasks (e.g. navigate to a cell and
row; transpose the table) and evaluate the performance differences when using 8
formats. In contrast to past work, we introduce 8 noise operations inspired by
real-world messy data and adversarial inputs, and show that such operations can
impact LLM performance across formats for different structural understanding
tasks.

通过引入基于自我监督任务的结构化任务集合以及噪声操作，本研究探讨大型语言模型在表格任务中的效能和结构理解能力。

LLMs 中的表格表示、噪声操作及其对表格结构理解任务的影响

Tabular Representation, Noisy Operators, and Impacts on Table Structure  Understanding Tasks in LLMs

Large language models (LLMs) are becoming attractive as few-shot reasoners to
solve NL-related tasks. However, there is still much to be learned about how
well LLMs understand structured data, such as tables. While it is true that
tables can be used as inputs to LLMs with serialization, there lack
comprehensive studies examining whether LLMs can truly comprehend such data. In
this paper we try to understand this by designing a benchmark to evaluate
structural understanding capabilities (SUC) of LLMs. The benchmark we create
includes seven tasks, each with their own unique challenges, e.g,, cell lookup,
row retrieval and size detection. We run a series of evaluations on GPT-3
family models (e.g., text-davinci-003). We discover that the performance varied
depending on a number of input choices, including table input format, content
order, role prompting and partition marks. Drawing from the insights gained
through the benchmark evaluations, we then propose self-augmentation for
effective structural prompting, e.g., critical value / range identification
using LLMs' internal knowledge. When combined with carefully chosen input
choices, these structural prompting methods lead to promising improvements in
LLM performance on a variety of tabular tasks, e.g., TabFact($\uparrow2.31\%$),
HybridQA($\uparrow2.13\%$), SQA($\uparrow2.72\%$), Feverous($\uparrow0.84\%$),
and ToTTo($\uparrow5.68\%$). We believe our benchmark and proposed prompting
methods can serve as a simple yet generic selection for future research. The
code and data are released in
this https URL

本文旨在通过设计一个基准来评估 LLMs 的结构理解能力，包括七个挑战不同的任务，使用 self-augmentation 方法来提高 LLMs 的表格任务表现，提高了 TabFact、HybridQA、SQA、Feverous 和 ToTTo 等表格任务的性能。