Textual information in a captured scene plays an important role in scene
interpretation and decision making. Though there exist methods that can
successfully detect and interpret complex text regions present in a scene, to
the best of our knowledge, there is no significant prior work that aims to
modify the textual information in an image. The ability to edit text directly
on images has several advantages including error correction, text restoration
and image reusability. In this paper, we propose a method to modify text in an
image at character-level. We approach the problem in two stages. At first, the
unobserved character (target) is generated from an observed character (source)
being modified. We propose two different neural network architectures - (a)
FANnet to achieve structural consistency with source font and (b) Colornet to
preserve source color. Next, we replace the source character with the generated
character maintaining both geometric and visual consistency with neighboring
characters. Our method works as a unified platform for modifying text in
images. We present the effectiveness of our method on COCO-Text and ICDAR
datasets both qualitatively and quantitatively.

本文介绍了一种用于在图像中修改文本的方法，该方法使用两种不同的神经网络体系结构生成目标字符，以实现透明性、一致性和准确性，旨在通过在图像上直接编辑文本来纠正错误、恢复文本和提高图像可重用性。

利用字体自适应神经网络的场景文本编辑器 STEFANN

STEFANN: Scene Text Editor using Font Adaptive Neural Network

Recent data-driven approaches to scene interpretation predominantly pose
inference as an end-to-end black-box mapping, commonly performed by a
Convolutional Neural Network (CNN). However, decades of work on perceptual
organization in both human and machine vision suggests that there are often
intermediate representations that are intrinsic to an inference task, and which
provide essential structure to improve generalization. In this work, we explore
an approach for injecting prior domain structure into neural network training
by supervising hidden layers of a CNN with intermediate concepts that normally
are not observed in practice. We formulate a probabilistic framework which
formalizes these notions and predicts improved generalization via this deep
supervision method. One advantage of this approach is that we are able to train
only from synthetic CAD renderings of cluttered scenes, where concept values
can be extracted, but apply the results to real images. Our implementation
achieves the state-of-the-art performance of 2D/3D keypoint localization and
image classification on real image benchmarks, including KITTI, PASCAL VOC,
PASCAL3D+, IKEA, and CIFAR100. We provide additional evidence that our approach
outperforms alternative forms of supervision, such as multi-task networks.

本文提出一种深度监督方法，通过利用中间概念对 CNN 的隐藏层进行监督，将先前的领域结构融入到神经网络训练中，显著提高了推广能力。该方法还可以仅使用合成场景来训练，并在实际图像上取得了 2D/3D 关键点本地化和图像分类的最新性能。