This paper presents IP-SLT, a simple yet effective framework for sign
language translation (SLT). Our IP-SLT adopts a recurrent structure and
enhances the semantic representation (prototype) of the input sign language
video via an iterative refinement manner. Our idea mimics the behavior of human
reading, where a sentence can be digested repeatedly, till reaching accurate
understanding. Technically, IP-SLT consists of feature extraction, prototype
initialization, and iterative prototype refinement. The initialization module
generates the initial prototype based on the visual feature extracted by the
feature extraction module. Then, the iterative refinement module leverages the
cross-attention mechanism to polish the previous prototype by aggregating it
with the original video feature. Through repeated refinement, the prototype
finally converges to a more stable and accurate state, leading to a fluent and
appropriate translation. In addition, to leverage the sequential dependence of
prototypes, we further propose an iterative distillation loss to compress the
knowledge of the final iteration into previous ones. As the autoregressive
decoding process is executed only once in inference, our IP-SLT is ready to
improve various SLT systems with acceptable overhead. Extensive experiments are
conducted on public benchmarks to demonstrate the effectiveness of the IP-SLT.

该论文介绍了 IP-SLT，这是一个简单而有效的手语翻译框架，通过迭代改善方式，增强了输入手语视频的语义表达（原型）。通过特征提取、原型初始化和迭代原型改进，IP-SLT 能够准确理解手语，并将其流畅和恰当地翻译成文本。