Understanding whether a generated table is of good quality is important to be
able to use it in creating or editing documents using automatic methods. In
this work, we underline that existing measures for table quality evaluation
fail to capture the overall semantics of the tables, and sometimes unfairly
penalize good tables and reward bad ones. We propose TabEval, a novel table
evaluation strategy that captures table semantics by first breaking down a
table into a list of natural language atomic statements and then compares them
with ground truth statements using entailment-based measures. To validate our
approach, we curate a dataset comprising of text descriptions for 1,250 diverse
Wikipedia tables, covering a range of topics and structures, in contrast to the
limited scope of existing datasets. We compare TabEval with existing metrics
using unsupervised and supervised text-to-table generation methods,
demonstrating its stronger correlation with human judgments of table quality
across four datasets.

通过 TabEval 方法，通过将表格转化为自然语言原子语句并使用蕴含度量对比真实语句，我们提出了一种新的表格评估策略，以捕捉表格的语义信息，并与现有方法进行比较，展示了其在人工判断表格质量方面具有更强的相关性。

这是一个糟糕的表格吗？从文本生成表格的评估再探讨

Is this a bad table? A Closer Look at the Evaluation of Table Generation  from Text

The task of condensing large chunks of textual information into concise and
structured tables has gained attention recently due to the emergence of Large
Language Models (LLMs) and their potential benefit for downstream tasks, such
as text summarization and text mining. Previous approaches often generate
tables that directly replicate information from the text, limiting their
applicability in broader contexts, as text-to-table generation in real-life
scenarios necessitates information extraction, reasoning, and integration.
However, there is a lack of both datasets and methodologies towards this task.
In this paper, we introduce LiveSum, a new benchmark dataset created for
generating summary tables of competitions based on real-time commentary texts.
We evaluate the performances of state-of-the-art LLMs on this task in both
fine-tuning and zero-shot settings, and additionally propose a novel pipeline
called $T^3$(Text-Tuple-Table) to improve their performances. Extensive
experimental results demonstrate that LLMs still struggle with this task even
after fine-tuning, while our approach can offer substantial performance gains
without explicit training. Further analyses demonstrate that our method
exhibits strong generalization abilities, surpassing previous approaches on
several other text-to-table datasets. Our code and data can be found at
this https URL

本论文介绍了 LiveSum，这是一个用于根据实时评论文本生成比赛摘要表格的新基准数据集。该论文评估了最先进的大型语言模型在这项任务中的性能，并提出了一种名为 $T^3$（文本 - 元组 - 表格）的新型流水线来改善性能。实验证明，即使在微调之后，大型语言模型在这项任务中仍然遇到困难，而我们的方法可以在没有显式训练的情况下获得显著的性能增益。进一步的分析表明，我们的方法具有较强的泛化能力，超过了以前的一些文本到表格数据集的方法。

文本 - 元组 - 表：通过全局元组提取实现文本到表生成中的信息整合

Text-Tuple-Table: Towards Information Integration in Text-to-Table  Generation via Global Tuple Extraction

Recently, the text-to-table generation task has attracted increasing
attention due to its wide applications. In this aspect, the dominant model
formalizes this task as a sequence-to-sequence generation task and serializes
each table into a token sequence during training by concatenating all rows in a
top-down order. However, it suffers from two serious defects: 1) the predefined
order introduces a wrong bias during training, which highly penalizes shifts in
the order between rows; 2) the error propagation problem becomes serious when
the model outputs a long token sequence. In this paper, we first conduct a
preliminary study to demonstrate the generation of most rows is
order-insensitive. Furthermore, we propose a novel sequence-to-sequence&set
text-to-table generation model. Specifically, in addition to a text encoder
encoding the input text, our model is equipped with a table header generator to
first output a table header, i.e., the first row of the table, in the manner of
sequence generation. Then we use a table body generator with learnable row
embeddings and column embeddings to generate a set of table body rows in
parallel. Particularly, to deal with the issue that there is no correspondence
between each generated table body row and target during training, we propose a
target assignment strategy based on the bipartite matching between the first
cells of generated table body rows and targets. Experiment results show that
our model significantly surpasses the baselines, achieving state-of-the-art
performance on commonly-used datasets.

本文介绍了一种用于文本到表格生成的新模型，该模型在考虑到多行中的大多数组合不敏感的情况下，首先采用文本编码器编码输入文本，再加入了一种表头生成器来以序列生成的方式输出表头，之后再使用具有可学习的行和列嵌入的表体生成器并行生成一组表身行，实验结果表明，此模型优于基线，达到了公认的数据集上的最佳性能。