Recent advanced methods in Natural Language Understanding for Task-oriented
Dialogue (TOD) Systems (e.g., intent detection and slot filling) require a
large amount of annotated data to achieve competitive performance. In reality,
token-level annotations (slot labels) are time-consuming and difficult to
acquire. In this work, we study the Slot Induction (SI) task whose objective is
to induce slot boundaries without explicit knowledge of token-level slot
annotations. We propose leveraging Unsupervised Pre-trained Language Model
(PLM) Probing and Contrastive Learning mechanism to exploit (1) unsupervised
semantic knowledge extracted from PLM, and (2) additional sentence-level intent
label signals available from TOD. Our approach is shown to be effective in SI
task and capable of bridging the gaps with token-level supervised models on two
NLU benchmark datasets. When generalized to emerging intents, our SI objectives
also provide enhanced slot label representations, leading to improved
performance on the Slot Filling tasks.

在本研究中，我们研究了无需显式标注的词级槽位注释的情况下，用于识别 Task-oriented Dialogue Systems 中的槽位边界的槽位感知（Slot Induction）任务，并提出利用无监督预训练语言模型（PLM）探测和对比学习机制，结合可从 Task-oriented Dialogue Systems 中获得的句级意图标签信号，从而在两个自然语言理解基准数据集上展现出有效性，同时能够填补与词级监督模型之间的差距。当推广到新兴意图时，我们的槽位感知目标还提供了改进的槽位标签表示，从而提高了槽位填充任务的性能。

通过预训练语言模型探测和多层对比学习实现槽位归纳

Slot Induction via Pre-trained Language Model Probing and Multi-level  Contrastive Learning

Text style transfer is the task of transferring the style of text having
certain stylistic attributes, while preserving non-stylistic or content
information. In this work we introduce the Generative Style Transformer (GST) -
a new approach to rewriting sentences to a target style in the absence of
parallel style corpora. GST leverages the power of both, large unsupervised
pre-trained language models as well as the Transformer. GST is a part of a
larger `Delete Retrieve Generate' framework, in which we also propose a novel
method of deleting style attributes from the source sentence by exploiting the
inner workings of the Transformer. Our models outperform state-of-art systems
across 5 datasets on sentiment, gender and political slant transfer. We also
propose the use of the GLEU metric as an automatic metric of evaluation of
style transfer, which we found to compare better with human ratings than the
predominantly used BLEU score.

本文提出了一种新的手段 —— 生成式转换器（GST），其使用无监督的预训练语言模型和 Transformer，以重写句子以达到目标风格的目的，并使用 GLEU 评估度量，超出其他当今最先进的情感、性别和政治审查数据集上的性能。