Extracting semantic information from generated text is a useful tool for
applications such as automated fact checking or retrieval augmented generation.
Currently, this requires either separate models during inference, which
increases computational cost, or destructive fine-tuning of the language model.
Instead, we propose directly embedding information extraction capabilities into
pre-trained language models using probing classifiers, enabling efficient
simultaneous text generation and information extraction. For this, we introduce
an approach called EMBER and show that it enables named entity recognition in
decoder-only language models without fine-tuning them and while incurring
minimal additional computational cost at inference time. Specifically, our
experiments using GPT-2 show that EMBER maintains high token generation rates
during streaming text generation, with only a negligible decrease in speed of
around 1% compared to a 43.64% slowdown measured for a baseline using a
separate NER model. Code and data are available at
this https URL

直接将信息提取功能嵌入预训练的语言模型中，使用探测分类器进行语义信息的提取，提高了文本生成和信息提取的效率，同时无需微调语言模型，内存所需较少且生成速率较高。

使用探测分类器的嵌入式命名实体识别

Embedded Named Entity Recognition using Probing Classifiers

This paper describes EMBER: a labeled benchmark dataset for training machine
learning models to statically detect malicious Windows portable executable
files. The dataset includes features extracted from 1.1M binary files: 900K
training samples (300K malicious, 300K benign, 300K unlabeled) and 200K test
samples (100K malicious, 100K benign). To accompany the dataset, we also
release open source code for extracting features from additional binaries so
that additional sample features can be appended to the dataset. This dataset
fills a void in the information security machine learning community: a
benign/malicious dataset that is large, open and general enough to cover
several interesting use cases. We enumerate several use cases that we
considered when structuring the dataset. Additionally, we demonstrate one use
case wherein we compare a baseline gradient boosted decision tree model trained
using LightGBM with default settings to MalConv, a recently published
end-to-end (featureless) deep learning model for malware detection. Results
show that even without hyper-parameter optimization, the baseline EMBER model
outperforms MalConv. The authors hope that the dataset, code and baseline model
provided by EMBER will help invigorate machine learning research for malware
detection, in much the same way that benchmark datasets have advanced computer
vision research.

EMBER 数据集为反恶意 Windows 便携式可执行文件静态检测训练机器学习模型提供标记的基准数据集，包括从 110 万个二进制文件中提取的特征：900K 个训练样本（300K 个恶意样本，300K 个良性样本，300K 个未标记样本）和 200K 个测试样本（100K 个恶意样本，100K 个良性样本），并提供相关的数据处理代码和基线模型 MalConv 性能比较结果。