The deep neural network (DNN) has been proven effective in various domains.
However, they often struggle to perform well on certain minority groups during
inference, despite showing strong performance on the majority of data groups.
This is because over-parameterized models learned \textit{bias attributes} from
a large number of \textit{bias-aligned} training samples. These bias attributes
are strongly spuriously correlated with the target variable, causing the models
to be biased towards spurious correlations (i.e., \textit{bias-conflicting}).
To tackle this issue, we propose a novel \textbf{re}weighted \textbf{s}parse
\textbf{t}raining framework, dubbed as \textit{\textbf{REST}}, which aims to
enhance the performance of biased data while improving computation and memory
efficiency. Our proposed REST framework has been experimentally validated on
three datasets, demonstrating its effectiveness in exploring unbiased
subnetworks. We found that REST reduces the reliance on spuriously correlated
features, leading to better performance across a wider range of data groups
with fewer training and inference resources. We highlight that the
\textit{REST} framework represents a promising approach for improving the
performance of DNNs on biased data, while simultaneously improving computation
and memory efficiency. By reducing the reliance on spurious correlations, REST
has the potential to enhance the robustness of DNNs and improve their
generalization capabilities. Code is released at
https://github.com/zhao1402072392/REST

通过重新权重稀疏训练的框架（REST），减少对偏见相关特征的依赖，改善深度神经网络（DNN）在有偏数据中的性能，同时提高计算和内存效率。