We introduce the structured scene-text spotting task, which requires a
scene-text OCR system to spot text in the wild according to a query regular
expression. Contrary to generic scene text OCR, structured scene-text spotting
seeks to dynamically condition both scene text detection and recognition on
user-provided regular expressions. To tackle this task, we propose the
Structured TExt sPotter (STEP), a model that exploits the provided text
structure to guide the OCR process. STEP is able to deal with regular
expressions that contain spaces and it is not bound to detection at the
word-level granularity. Our approach enables accurate zero-shot structured text
spotting in a wide variety of real-world reading scenarios and is solely
trained on publicly available data. To demonstrate the effectiveness of our
approach, we introduce a new challenging test dataset that contains several
types of out-of-vocabulary structured text, reflecting important reading
applications of fields such as prices, dates, serial numbers, license plates
etc. We demonstrate that STEP can provide specialised OCR performance on demand
in all tested scenarios.

我们介绍了结构化场景文本定位任务，该任务要求场景文本 OCR 系统根据查询正则表达式在野外定位文本。为了解决这个任务，我们提出了 Structured TExt sPotter（STEP）模型，该模型利用提供的文本结构来指导 OCR 过程。我们的方法能够准确地在各种实际阅读场景中零样本结构化文本定位，并且仅使用公开可用的数据进行训练。

STEP -- 面向结构化场景文本检测

STEP -- Towards Structured Scene-Text Spotting

We introduce a simple new approach to the problem of understanding documents
where non-trivial layout influences the local semantics. To this end, we modify
the Transformer encoder architecture in a way that allows it to use layout
features obtained from an OCR system, without the need to re-learn language
semantics from scratch. We only augment the input of the model with the
coordinates of token bounding boxes, avoiding, in this way, the use of raw
images. This leads to a layout-aware language model which can then be
fine-tuned on downstream tasks.
The model is evaluated on an end-to-end information extraction task using
four publicly available datasets: Kleister NDA, Kleister Charity, SROIE and
CORD. We show that our model achieves superior performance on datasets
consisting of visually rich documents, while also outperforming the baseline
RoBERTa on documents with flat layout (NDA \(F_{1}\) increase from 78.50 to
80.42). Our solution ranked first on the public leaderboard for the Key
Information Extraction from the SROIE dataset, improving the SOTA
\(F_{1}\)-score from 97.81 to 98.17.

在 OCR 系统获取的布局特征的基础上，改进 Transformer 编码器的结构来构建布局感知的语言模型，展示在信息提取方面的出色性能。