Understanding whether a generated table is of good quality is important to be
able to use it in creating or editing documents using automatic methods. In
this work, we underline that existing measures for table quality evaluation
fail to capture the overall semantics of the tables, and sometimes unfairly
penalize good tables and reward bad ones. We propose TabEval, a novel table
evaluation strategy that captures table semantics by first breaking down a
table into a list of natural language atomic statements and then compares them
with ground truth statements using entailment-based measures. To validate our
approach, we curate a dataset comprising of text descriptions for 1,250 diverse
Wikipedia tables, covering a range of topics and structures, in contrast to the
limited scope of existing datasets. We compare TabEval with existing metrics
using unsupervised and supervised text-to-table generation methods,
demonstrating its stronger correlation with human judgments of table quality
across four datasets.

通过 TabEval 方法，通过将表格转化为自然语言原子语句并使用蕴含度量对比真实语句，我们提出了一种新的表格评估策略，以捕捉表格的语义信息，并与现有方法进行比较，展示了其在人工判断表格质量方面具有更强的相关性。