Unified Sequence Labeling that articulates different sequence labeling
problems such as Named Entity Recognition, Relation Extraction, Semantic Role
Labeling, etc. in a generalized sequence-to-sequence format opens up the
opportunity to make the maximum utilization of large language model knowledge
toward structured prediction. Unfortunately, this requires formatting them into
specialized augmented format unknown to the base pretrained language model
(PLMs) necessitating finetuning to the target format. This significantly bounds
its usefulness in data-limited settings where finetuning large models cannot
properly generalize to the target format. To address this challenge and
leverage PLM knowledge effectively, we propose FISH-DIP, a sample-aware dynamic
sparse finetuning strategy that selectively focuses on a fraction of
parameters, informed by feedback from highly regressing examples, during the
fine-tuning process. By leveraging the dynamism of sparsity, our approach
mitigates the impact of well-learned samples and prioritizes underperforming
instances for improvement in generalization. Across five tasks of sequence
labeling, we demonstrate that FISH-DIP can smoothly optimize the model in low
resource settings offering upto 40% performance improvements over full
fine-tuning depending on target evaluation settings. Also, compared to
in-context learning and other parameter-efficient fine-tuning approaches,
FISH-DIP performs comparably or better, notably in extreme low-resource
settings.

采用 FISH-DIP 方法，通过在低资源环境下优化模型并在序列标注任务中取得高达 40％的性能提升，成功地解决了基于大语言模型的序列标注问题中受限于数据量问题的挑战。

样本感知动态稀疏精调的统一低资源序列标注

Unified Low-Resource Sequence Labeling by Sample-Aware Dynamic Sparse  Finetuning

Both few-shot and continual learning have seen substantial progress in the
last years due to the introduction of proper benchmarks. That being said, the
field has still to frame a suite of benchmarks for the highly desirable setting
of continual few-shot learning, where the learner is presented a number of
few-shot tasks, one after the other, and then asked to perform well on a
validation set stemming from all previously seen tasks. Continual few-shot
learning has a small computational footprint and is thus an excellent setting
for efficient investigation and experimentation. In this paper we first define
a theoretical framework for continual few-shot learning, taking into account
recent literature, then we propose a range of flexible benchmarks that unify
the evaluation criteria and allows exploring the problem from multiple
perspectives. As part of the benchmark, we introduce a compact variant of
ImageNet, called SlimageNet64, which retains all original 1000 classes but only
contains 200 instances of each one (a total of 200K data-points) downscaled to
64 x 64 pixels. We provide baselines for the proposed benchmarks using a number
of popular few-shot learning algorithms, as a result, exposing previously
unknown strengths and weaknesses of those algorithms in continual and
data-limited settings.

该研究为连续性小样本学习引入了一套灵活的基准测试框架 ——SlimageNet64，并使用流行的少样本学习算法提供了基线，以暴露这些算法在连续和数据受限情况下的优点和缺点。