This study addresses the application of encoder-only Pre-trained Language
Models (PLMs) in keyphrase generation (KPG) amidst the broader availability of
domain-tailored encoder-only models compared to encoder-decoder models. We
investigate three core inquiries: (1) the efficacy of encoder-only PLMs in KPG,
(2) optimal architectural decisions for employing encoder-only PLMs in KPG, and
(3) a performance comparison between in-domain encoder-only and encoder-decoder
PLMs across varied resource settings. Our findings, derived from extensive
experimentation in two domains reveal that with encoder-only PLMs, although KPE
with Conditional Random Fields slightly excels in identifying present
keyphrases, the KPG formulation renders a broader spectrum of keyphrase
predictions. Additionally, prefix-LM fine-tuning of encoder-only PLMs emerges
as a strong and data-efficient strategy for KPG, outperforming general-domain
seq2seq PLMs. We also identify a favorable parameter allocation towards model
depth rather than width when employing encoder-decoder architectures
initialized with encoder-only PLMs. The study sheds light on the potential of
utilizing encoder-only PLMs for advancing KPG systems and provides a groundwork
for future KPG methods. Our code and pre-trained checkpoints are released at
this https URL

该研究探讨了仅具有编码器的预训练语言模型在关键词生成中的应用，研究了优化架构决策以及对资源设置进行性能比较的效果。研究发现，使用仅具有编码器的预训练语言模型进行关键词生成能够广泛预测关键词，而使用前缀语言模型对编码器进行微调则是一种强大且高效的策略。该研究对推动关键词生成系统的发展提供了有价值的见解，并为未来的研究方法奠定了基础。

利用仅编码器预训练语言模型进行有效关键词生成

On Leveraging Encoder-only Pre-trained Language Models for Effective  Keyphrase Generation

Standard methods for multi-label text classification largely rely on
encoder-only pre-trained language models, whereas encoder-decoder models have
proven more effective in other classification tasks. In this study, we compare
four methods for multi-label classification, two based on an encoder only, and
two based on an encoder-decoder. We carry out experiments on four datasets -two
in the legal domain and two in the biomedical domain, each with two levels of
label granularity- and always depart from the same pre-trained model, T5. Our
results show that encoder-decoder methods outperform encoder-only methods, with
a growing advantage on more complex datasets and labeling schemes of finer
granularity. Using encoder-decoder models in a non-autoregressive fashion, in
particular, yields the best performance overall, so we further study this
approach through ablations to better understand its strengths.

本研究比较了四种多标签分类方法，其中两种基于编码器，两种基于编码器 - 解码器。结果表明，在多个数据集上使用编码器 - 解码器方法比仅编码器表现更好，其非自回归编码器 - 解码器方法获得了最优表现。