We consider the task of few-shot intent detection, which involves training a
deep learning model to classify utterances based on their underlying intents
using only a small amount of labeled data. The current approach to address this
problem is through continual pre-training, i.e., fine-tuning pre-trained
language models (PLMs) on external resources (e.g., conversational corpora,
public intent detection datasets, or natural language understanding datasets)
before using them as utterance encoders for training an intent classifier. In
this paper, we show that continual pre-training may not be essential, since the
overfitting problem of PLMs on this task may not be as serious as expected.
Specifically, we find that directly fine-tuning PLMs on only a handful of
labeled examples already yields decent results compared to methods that employ
continual pre-training, and the performance gap diminishes rapidly as the
number of labeled data increases. To maximize the utilization of the limited
available data, we propose a context augmentation method and leverage
sequential self-distillation to boost performance. Comprehensive experiments on
real-world benchmarks show that given only two or more labeled samples per
class, direct fine-tuning outperforms many strong baselines that utilize
external data sources for continual pre-training. The code can be found at
this https URL

本文针对少样本意图检测任务进行研究，提出一种新的方法，即直接在少量标记数据上对预训练语言模型（PLMs）进行微调，使用上下文增强方法和序列自蒸馏技术进一步优化性能。实验表明，该方法优于许多强基线方法，甚至是在只有每类两个或更多标记样本的情况下。

使用 PLMs 重新审视少样本意图分类：直接微调 vs. 连续预训练

Revisit Few-shot Intent Classification with PLMs: Direct Fine-tuning vs.  Continual Pre-training

Limited by the locality of convolutional neural networks, most existing local
features description methods only learn local descriptors with local
information and lack awareness of global and surrounding spatial context. In
this work, we focus on making local descriptors "look wider to describe better"
by learning local Descriptors with More Than just Local information (MTLDesc).
Specifically, we resort to context augmentation and spatial attention
mechanisms to make our MTLDesc obtain non-local awareness. First, Adaptive
Global Context Augmented Module and Diverse Local Context Augmented Module are
proposed to construct robust local descriptors with context information from
global to local. Second, Consistent Attention Weighted Triplet Loss is designed
to integrate spatial attention awareness into both optimization and matching
stages of local descriptors learning. Third, Local Features Detection with
Feature Pyramid is given to obtain more stable and accurate keypoints
localization. With the above innovations, the performance of our MTLDesc
significantly surpasses the prior state-of-the-art local descriptors on
HPatches, Aachen Day-Night localization and InLoc indoor localization
benchmarks.

通过采用上下文增强和空间注意机制，本文提出了一种可获得非局部感知的本地描述符方法，并在诸如 HPatches，Aachen Day-Night 定位和 InLoc 室内定位基准测试中，显著超越了先前的最先进本地描述符。