We describe a simple and effective method (Spectral Attribute removaL; SAL)
to remove private or guarded information from neural representations. Our
method uses matrix decomposition to project the input representations into
directions with reduced covariance with the guarded information rather than
maximal covariance as factorization methods normally use. We begin with linear
information removal and proceed to generalize our algorithm to the case of
nonlinear information removal using kernels. Our experiments demonstrate that
our algorithm retains better main task performance after removing the guarded
information compared to previous work. In addition, our experiments demonstrate
that we need a relatively small amount of guarded attribute data to remove
information about these attributes, which lowers the exposure to sensitive data
and is more suitable for low-resource scenarios. Code is available at
this https URL.

该研究介绍了一种简单有效的方法，称为 SAL，它使用矩阵分解将输入向量投影到与守卫信息减少协方差的方向上，从而达到去除神经表示中的私密信息的目的。这种方法也适用于非线性信息去除，且其实验结果表明该算法将主要任务性能与去除守卫信息后的性能相比较，更优秀，同时只需要使用相对较少量的守卫信息即可，更适用于低资源的情境。