We present a neural encoder-decoder model to convert images into
presentational markup based on a scalable coarse-to-fine attention mechanism.
Our method is evaluated in the context of image-to-LaTeX generation, and we
introduce a new dataset of real-world rendered mathematical expressions paired
with LaTeX markup. We show that unlike neural OCR techniques using CTC-based
models, attention-based approaches can tackle this non-standard OCR task. Our
approach outperforms classical mathematical OCR systems by a large margin on
in-domain rendered data, and, with pretraining, also performs well on
out-of-domain handwritten data. To reduce the inference complexity associated
with the attention-based approaches, we introduce a new coarse-to-fine
attention layer that selects a support region before applying attention.

该研究提出一种神经编码解码模型，采用可扩展的粗 - 精细注意机制将图片转化为表现标记，并在图像到 LaTeX 生成的上下文中进行评估。该方法通过引入真实世界渲染的数学表达式与 LaTeX 标记相配对的新数据集，展示了与 CTC 模型相比采用基于注意的方法可以处理非标准 OCR 任务，并在域内渲染数据上远优于经典的数字 OCR 系统，在预训练的情况下，该方法也可以在域外手写数据上表现良好。为了减少与基于注意力的方法相关的推理复杂度，该研究还引入了一个选择支持区域后应用注意力的新的粗 - 精细注意力层。