Existing techniques for text detection can be broadly classified into two
primary groups: segmentation-based methods and regression-based methods.
Segmentation models offer enhanced robustness to font variations but require
intricate post-processing, leading to high computational overhead.
Regression-based methods undertake instance-aware prediction but face
limitations in robustness and data efficiency due to their reliance on
high-level representations. In our academic pursuit, we propose SRFormer, a
unified DETR-based model with amalgamated Segmentation and Regression, aiming
at the synergistic harnessing of the inherent robustness in segmentation
representations, along with the straightforward post-processing of
instance-level regression. Our empirical analysis indicates that favorable
segmentation predictions can be obtained at the initial decoder layers. In
light of this, we constrain the incorporation of segmentation branches to the
first few decoder layers and employ progressive regression refinement in
subsequent layers, achieving performance gains while minimizing additional
computational load from the mask. Furthermore, we propose a Mask-informed Query
Enhancement module. We take the segmentation result as a natural soft-ROI to
pool and extract robust pixel representations, which are then employed to
enhance and diversify instance queries. Extensive experimentation across
multiple benchmarks has yielded compelling findings, highlighting our method's
exceptional robustness, superior training and data efficiency, as well as its
state-of-the-art performance.

我们提出了一种基于 SRFormer 的统一模型，融合了分割和回归，旨在实现分割表示的鲁棒性和实例级回归的直接后处理。通过在初始解码器层获取有利的分割预测结果，我们限制了分割分支的融入，并在后续层中采用渐进式回归优化，实现了性能的提升，同时最小化了掩膜带来的额外计算负载。此外，我们提出了一个基于掩膜信息的查询增强模块，利用分割结果作为自然软区域感兴趣池化和提取鲁棒的像素表示，进而增强和多样化实例查询。在多个基准测试中进行了广泛实验，得出了有说服力的结果，突显了我们的方法在鲁棒性、训练和数据效率以及性能方面的卓越表现。

SRFormer：基于回归的文本检测 Transformer 与分割技术的协同增强

SRFormer: Empowering Regression-Based Text Detection Transformer with  Segmentation

Recently, segmentation-based methods are quite popular in scene text
detection, as the segmentation results can more accurately describe scene text
of various shapes such as curve text. However, the post-processing of
binarization is essential for segmentation-based detection, which converts
probability maps produced by a segmentation method into bounding boxes/regions
of text. In this paper, we propose a module named Differentiable Binarization
(DB), which can perform the binarization process in a segmentation network.
Optimized along with a DB module, a segmentation network can adaptively set the
thresholds for binarization, which not only simplifies the post-processing but
also enhances the performance of text detection. Based on a simple segmentation
network, we validate the performance improvements of DB on five benchmark
datasets, which consistently achieves state-of-the-art results, in terms of
both detection accuracy and speed. In particular, with a light-weight backbone,
the performance improvements by DB are significant so that we can look for an
ideal tradeoff between detection accuracy and efficiency. Specifically, with a
backbone of ResNet-18, our detector achieves an F-measure of 82.8, running at
62 FPS, on the MSRA-TD500 dataset. Code is available at:
this https URL

本文提出了一种名为 Differentiable Binarization（DB）的模块，可以在分割网络中执行二值化过程，不仅简化了后期处理，还提高了文本检测的性能。使用 ResNet-18 为 backbone，在 MSRA-TD500 数据集上实现了 82.8 的 F-measure，达到 62 FPS 的速度。