Automated evaluation of text generation systems has recently seen increasing
attention, particularly checking whether generated text stays truthful to input
sources. Existing methods frequently rely on an evaluation using task-specific
language models, which in turn allows for little interpretability of generated
scores. We introduce SRLScore, a reference-free evaluation metric designed with
text summarization in mind. Our approach generates fact tuples constructed from
Semantic Role Labels, applied to both input and summary texts. A final
factuality score is computed by an adjustable scoring mechanism, which allows
for easy adaption of the method across domains. Correlation with human
judgments on English summarization datasets shows that SRLScore is competitive
with state-of-the-art methods and exhibits stable generalization across
datasets without requiring further training or hyperparameter tuning. We
experiment with an optional co-reference resolution step, but find that the
performance boost is mostly outweighed by the additional compute required. Our
metric is available online at this https URL

SRLScore 是一种基于语义角色标注的自动文本评估度量，适用于文本摘要，能显示出稳定的泛化性能，且可适应于不同的领域。

使用语义角色标注评估文本的事实一致性

Evaluating Factual Consistency of Texts with Semantic Role Labeling

Machine Reading Comprehension has become one of the most advanced and popular
research topics in the fields of Natural Language Processing in recent years.
The classification of answerability questions is a relatively significant
sub-task in machine reading comprehension; however, there haven't been many
studies. Retro-Reader is one of the studies that has solved this problem
effectively. However, the encoders of most traditional machine reading
comprehension models in general and Retro-Reader, in particular, have not been
able to exploit the contextual semantic information of the context completely.
Inspired by SemBERT, we use semantic role labels from the SRL task to add
semantics to pre-trained language models such as mBERT, XLM-R, PhoBERT. This
experiment was conducted to compare the influence of semantics on the
classification of answerability for the Vietnamese machine reading
comprehension. Additionally, we hope this experiment will enhance the encoder
for the Retro-Reader model's Sketchy Reading Module. The improved Retro-Reader
model's encoder with semantics was first applied to the Vietnamese Machine
Reading Comprehension task and obtained positive results.

本篇研究报告讲述 SemSRL 应用在预训练语言模型中以提高 Retro-Reader 在越南语阅读理解任务上的性能表现