Large language models (LLMs) have shown impressive capabilities in adapting
to various tasks when provided with task-specific instructions. However, LLMs
using standard decoding strategies often struggle with deviations from the
inputs. Intuitively, compliant LLM outputs should reflect the information
present in the input, which can be measured by point-wise mutual information
(PMI) scores. Therefore, we propose Diver, a novel approach that enhances LLM
Decoding through span-level PMI verification. During inference, Diver first
identifies divergence steps that may lead to multiple candidate spans.
Subsequently, it calculates the PMI scores by assessing the log-likelihood
gains of the input if the candidate spans are generated. Finally, the optimal
span is selected based on the PMI re-ranked output distributions. We evaluate
our method across various downstream tasks, and empirical results demonstrate
that Diver significantly outperforms existing decoding methods in both
performance and versatility.

使用点对点的互信息评分，我们提出了一种增强大型语言模型解码的新方法：Diver。实验证明，Diver 在性能和适用性方面明显优于现有的解码方法。

Diver: 使用基于跨度级互信息验证的大型语言模型解码

Diver: Large Language Model Decoding with Span-Level Mutual Information  Verification

As a fundamental natural language processing task and one of core knowledge
extraction techniques, named entity recognition (NER) is widely used to extract
information from texts for downstream tasks. Nested NER is a branch of NER in
which the named entities (NEs) are nested with each other. However, most of the
previous studies on nested NER usually apply linear structure to model the
nested NEs which are actually accommodated in a hierarchical structure. Thus in
order to address this mismatch, this work models the full nested NEs in a
sentence as a holistic structure, then we propose a holistic structure parsing
algorithm to disclose the entire NEs once for all. Besides, there is no
research on applying corpus-level information to NER currently. To make up for
the loss of this information, we introduce Point-wise Mutual Information (PMI)
and other frequency features from corpus-aware statistics for even better
performance by holistic modeling from sentence-level to corpus-level.
Experiments show that our model yields promising results on widely-used
benchmarks which approach or even achieve state-of-the-art. Further empirical
studies show that our proposed corpus-aware features can substantially improve
NER domain adaptation, which demonstrates the surprising advantage of our
proposed corpus-level holistic structure modeling.

本文介绍了一种针对命名实体识别中嵌套实体的一种新方法 —— 全局结构分析和点互信息统计，提高了该方法的性能及领域适应能力。

基于整体结构分析的嵌套命名实体识别

Nested Named Entity Recognition as Holistic Structure Parsing

Although many context-aware neural machine translation models have been
proposed to incorporate contexts in translation, most of those models are
trained end-to-end on parallel documents aligned in sentence-level. Because
only a few domains (and language pairs) have such document-level parallel data,
we cannot perform accurate context-aware translation in most domains. We
therefore present a simple method to turn a sentence-level translation model
into a context-aware model by incorporating a document-level language model
into the decoder. Our context-aware decoder is built upon only a sentence-level
parallel corpora and monolingual corpora; thus no document-level parallel data
is needed. In a theoretical viewpoint, the core part of this work is the novel
representation of contextual information using point-wise mutual information
between context and the current sentence. We show the effectiveness of our
approach in three language pairs, English to French, English to Russian, and
Japanese to English, by evaluation in \textsc{bleu} and contrastive tests for
context-aware translation.

该研究提出了一种简单的方法将基于句子水平的翻译模型转化为基于上下文的模型，该方法使用点互信息来表示上下文信息。这种方法在三种语言对上得到了较好的效果。