Several works have proven that finetuning is an applicable approach for
debiasing contextualized word embeddings. Similarly, discrete prompts with
semantic meanings have shown to be effective in debiasing tasks. With unfixed
mathematical representation at the token level, continuous prompts usually
surpass discrete ones at providing a pre-trained language model (PLM) with
additional task-specific information. Despite this, relatively few efforts have
been made to debias PLMs by prompt tuning with continuous prompts compared to
its discrete counterpart. Furthermore, for most debiasing methods that alter a
PLM's original parameters, a major problem is the need to not only decrease the
bias in the PLM but also to ensure that the PLM does not lose its
representation ability. Finetuning methods typically have a hard time
maintaining this balance, as they tend to violently remove meanings of
attribute words. In this paper, we propose ADEPT, a method to debias PLMs using
prompt tuning while maintaining the delicate balance between removing biases
and ensuring representation ability. To achieve this, we propose a new training
criterion inspired by manifold learning and equip it with an explicit debiasing
term to optimize prompt tuning. In addition, we conduct several experiments
with regard to the reliability, quality, and quantity of a previously proposed
attribute training corpus in order to obtain a clearer prototype of a certain
attribute, which indicates the attribute's position and relative distances to
other words on the manifold. We evaluate ADEPT on several widely acknowledged
debiasing benchmarks and downstream tasks, and find that it achieves
competitive results while maintaining (and in some cases even improving) the
PLM's representation ability. We further visualize words' correlation before
and after debiasing a PLM, and give some possible explanations for the visible
effects.

ADEPT 是一种基于流形学习的用于去偏见的 pre-trained language model（PLM）的方法，具有显式的去偏见术语来优化提示调整，旨在在保持平衡的同时维护 PLM 的表征能力。该方法通过多个实验在多个广泛认可的去偏见基准和下游任务上得到了有效的应用。

ADEPT：一个去偏见的提示框架

ADEPT: A DEbiasing PrompT Framework

Existing action localization approaches adopt shallow temporal convolutional
networks (\ie, TCN) on 1D feature map extracted from video frames. In this
paper, we empirically find that stacking more conventional temporal convolution
layers actually deteriorates action classification performance, possibly
ascribing to that all channels of 1D feature map, which generally are highly
abstract and can be regarded as latent concepts, are excessively recombined in
temporal convolution. To address this issue, we introduce a novel concept-wise
temporal convolution (CTC) layer as an alternative to conventional temporal
convolution layer for training deeper action localization networks. Instead of
recombining latent concepts, CTC layer deploys a number of temporal filters to
each concept separately with shared filter parameters across concepts. Thus can
capture common temporal patterns of different concepts and significantly enrich
representation ability. Via stacking CTC layers, we proposed a deep
concept-wise temporal convolutional network (C-TCN), which boosts the
state-of-the-art action localization performance on THUMOS'14 from 42.8 to 52.1
in terms of mAP(\%), achieving a relative improvement of 21.7\%. Favorable
result is also obtained on ActivityNet.

本文提出了一种概念级别的时间卷积层（CTC），用于训练深度行动本地化神经网络，通过在每个概念上分别使用多个时间滤波器捕捉不同概念的共同时间模式，极大地丰富了表示能力，通过堆叠 CTC 层，提出了一种深度概念级别的时间卷积神经网络（C-TCN），在 THUMOS'14 上将 mAP (%) 从 42.8 提高到 52.1，相对提高了 21.7\%。