As generated text becomes more commonplace, it is increasingly important to
evaluate how well-supported such text is by external knowledge sources. Many
approaches for evaluating textual support rely on some method for decomposing
text into its individual subclaims which are scored against a trusted
reference. We investigate how various methods of claim decomposition --
especially LLM-based methods -- affect the result of an evaluation approach
such as the recently proposed FActScore, finding that it is sensitive to the
decomposition method used. This sensitivity arises because such metrics
attribute overall textual support to the model that generated the text even
though error can also come from the metric's decomposition step. To measure
decomposition quality, we introduce an adaptation of FActScore, which we call
DecompScore. We then propose an LLM-based approach to generating decompositions
inspired by Bertrand Russell's theory of logical atomism and neo-Davidsonian
semantics and demonstrate its improved decomposition quality over previous
methods.

研究探讨了生成文本的外部知识支持度评估方法，发现该评估方法对于存证方法的选择非常敏感，并提出了一种改进方法来提高分解质量。