Understanding event descriptions is a central aspect of language processing,
but current approaches focus overwhelmingly on single sentences or documents.
Aggregating information about an event \emph{across documents} can offer a much
richer understanding. To this end, we present FAMuS, a new corpus of Wikipedia
passages that \emph{report} on some event, paired with underlying,
genre-diverse (non-Wikipedia) \emph{source} articles for the same event. Events
and (cross-sentence) arguments in both report and source are annotated against
FrameNet, providing broad coverage of different event types. We present results
on two key event understanding tasks enabled by FAMuS: \emph{source validation}
-- determining whether a document is a valid source for a target report event
-- and \emph{cross-document argument extraction} -- full-document argument
extraction for a target event from both its report and the correct source
article. We release both FAMuS and our models to support further research.

理解事件描述是语言处理的核心方面，但目前的方法主要集中在单个句子或文档上。跨文档汇总事件信息能提供更丰富的理解。为此，我们介绍了 FAMuS，这是一个新的维基百科段落语料库，用于报告某个事件，并提供与之对应的不同体裁（非维基百科）的源文章。我们对报告和源文章中的事件和跨句子的论证进行了 FrameNet 注释，提供了不同事件类型的广泛覆盖。我们展示了 FAMuS 支持的两个关键事件理解任务的结果：源验证 —— 确定一篇文档是否是目标报告事件的有效来源，以及跨文档的论证提取 —— 从报告和正确的源文章中提取目标事件的全文档论证。我们通过发布 FAMuS 和我们的模型来支持进一步研究。

FAMuS：多来源的帧

FAMuS: Frames Across Multiple Sources

People constantly use language to learn about the world. Computational
linguists have capitalized on this fact to build large language models (LLMs)
that acquire co-occurrence-based knowledge from language corpora. LLMs achieve
impressive performance on many tasks, but the robustness of their world
knowledge has been questioned. Here, we ask: do LLMs acquire generalized
knowledge about real-world events? Using curated sets of minimal sentence pairs
(n=1215), we tested whether LLMs are more likely to generate plausible event
descriptions compared to their implausible counterparts. We found that LLMs
systematically distinguish possible and impossible events (The teacher bought
the laptop vs. The laptop bought the teacher) but fall short of human
performance when distinguishing likely and unlikely events (The nanny tutored
the boy vs. The boy tutored the nanny). In follow-up analyses, we show that (i)
LLM scores are driven by both plausibility and surface-level sentence features,
(ii) LLMs generalize well across syntactic sentence variants (active vs
passive) but less well across semantic sentence variants (synonymous
sentences), (iii) some, but not all LLM deviations from ground-truth labels
align with crowdsourced human judgments, and (iv) explicit event plausibility
information emerges in middle LLM layers and remains high thereafter. Overall,
our analyses reveal a gap in LLMs' event knowledge, highlighting their
limitations as generalized knowledge bases. We conclude by speculating that the
differential performance on impossible vs. unlikely events is not a temporary
setback but an inherent property of LLMs, reflecting a fundamental difference
between linguistic knowledge and world knowledge in intelligent systems.

研究人员测试了大型语言模型是否能够生成可信的真实世界事件描述，并发现这些模型虽然能够区分可能和不可能的事件，但在区分可能性较小的事件时仍然比人类表现差，这突显了 LLM 在事件知识方面存在的局限性。

大型语言模型中的事件知识：不可能与不太可能之间的差距

Event knowledge in large language models: the gap between the impossible and the unlikely

We present a task of multilingual linking of events to a knowledge base. We
automatically compile a large-scale dataset for this task, comprising of 1.8M
mentions across 44 languages referring to over 10.9K events from Wikidata. We
propose two variants of the event linking task: 1) multilingual, where event
descriptions are from the same language as the mention, and 2) crosslingual,
where all event descriptions are in English. On the two proposed tasks, we
compare multiple event linking systems including BM25+ (Lv and Zhai, 2011) and
multilingual adaptations of the biencoder and crossencoder architectures from
BLINK (Wu et al., 2020). In our experiments on the two task variants, we find
both biencoder and crossencoder models significantly outperform the BM25+
baseline. Our results also indicate that the crosslingual task is in general
more challenging than the multilingual task. To test the out-of-domain
generalization of the proposed linking systems, we additionally create a
Wikinews-based evaluation set. We present qualitative analysis highlighting
various aspects captured by the proposed dataset, including the need for
temporal reasoning over context and tackling diverse event descriptions across
languages.

本文提出了一项多语种事件链接的任务，构建了包含 440 万触及 440 多种语言的事件数据集，其中两个变体均超过了基线模型。