Automating sign language translation (SLT) is a challenging real world
application. Despite its societal importance, though, research progress in the
field remains rather poor. Crucially, existing methods that yield viable
performance necessitate the availability of laborious to obtain gloss sequence
groundtruth. In this paper, we attenuate this need, by introducing an
end-to-end SLT model that does not entail explicit use of glosses; the model
only needs text groundtruth. This is in stark contrast to existing end-to-end
models that use gloss sequence groundtruth, either in the form of a modality
that is recognized at an intermediate model stage, or in the form of a parallel
output process, jointly trained with the SLT model. Our approach constitutes a
Transformer network with a novel type of layers that combines: (i) local
winner-takes-all (LWTA) layers with stochastic winner sampling, instead of
conventional ReLU layers, (ii) stochastic weights with posterior distributions
estimated via variational inference, and (iii) a weight compression technique
at inference time that exploits estimated posterior variance to perform
massive, almost lossless compression. We demonstrate that our approach can
reach the currently best reported BLEU-4 score on the PHOENIX 2014T benchmark,
but without making use of glosses for model training, and with a memory
footprint reduced by more than 70%.

本文提出一种端到端模型，不需要手势序列，只使用文本，其由 Transformer 网络和一种新型的局部赢者采取所有层组成，并演示了其在 PHOENIX 2014T 基准测试中达到了最佳 BLEU-4 分数，但没有使用手势进行模型训练，并且内存占用减少了 70％以上。