A major impediment to the advancement of sign language translation (SLT) is
data scarcity. Much of the sign language data currently available on the web
cannot be used for training supervised models due to the lack of aligned
captions. Furthermore, scaling SLT using large-scale web-scraped datasets bears
privacy risks due to the presence of biometric information, which the
responsible development of SLT technologies should account for. In this work,
we propose a two-stage framework for privacy-aware SLT at scale that addresses
both of these issues. We introduce SSVP-SLT, which leverages self-supervised
video pretraining on anonymized and unannotated videos, followed by supervised
SLT finetuning on a curated parallel dataset. SSVP-SLT achieves
state-of-the-art finetuned and zero-shot gloss-free SLT performance on the
How2Sign dataset, outperforming the strongest respective baselines by over 3
BLEU-4. Based on controlled experiments, we further discuss the advantages and
limitations of self-supervised pretraining and anonymization via facial
obfuscation for SLT.

通过两阶段的框架提出了 SSVP-SLT，该框架应对了缺乏对齐字幕的手语数据的问题，并解决了基于大规模网络抓取的数据集存在的隐私风险，通过自监督视频预训练和面部模糊化来提高 SLT 性能和保护隐私。