We present an efficient foveal framework to perform object detection. A scale
normalized image pyramid (SNIP) is generated that, like human vision, only
attends to objects within a fixed size range at different scales. Such a
restriction of objects' size during training affords better learning of
object-sensitive filters, and therefore, results in better accuracy. However,
the use of an image pyramid increases the computational cost. Hence, we propose
an efficient spatial sub-sampling scheme which only operates on fixed-size
sub-regions likely to contain objects (as object locations are known during
training). The resulting approach, referred to as Scale Normalized Image
Pyramid with Efficient Resampling or SNIPER, yields up to 3 times speed-up
during training. Unfortunately, as object locations are unknown during
inference, the entire image pyramid still needs processing. To this end, we
adopt a coarse-to-fine approach, and predict the locations and extent of
object-like regions which will be processed in successive scales of the image
pyramid. Intuitively, it's akin to our active human-vision that first skims
over the field-of-view to spot interesting regions for further processing and
only recognizes objects at the right resolution. The resulting algorithm is
referred to as AutoFocus and results in a 2.5-5 times speed-up during inference
when used with SNIP.

该论文提出了一种有效的点视觉框架，用于物体检测，其使用尺度归一化的图像金字塔（SNIP）生成已知训练期间可能包含物体的固定尺寸子区域，以提高性能，并通过粗到精的逐步缩小图像金字塔中的感兴趣区域的方法，快速识别物体。

带有自动对焦的标准化尺度图像金字塔用于目标检测

Scale Normalized Image Pyramids with AutoFocus for Object Detection

Autofocus is an important task for digital cameras, yet current approaches
often exhibit poor performance. We propose a learning-based approach to this
problem, and provide a realistic dataset of sufficient size for effective
learning. Our dataset is labeled with per-pixel depths obtained from multi-view
stereo, following "Learning single camera depth estimation using dual-pixels".
Using this dataset, we apply modern deep classification models and an ordinal
regression loss to obtain an efficient learning-based autofocus technique. We
demonstrate that our approach provides a significant improvement compared with
previous learned and non-learned methods: our model reduces the mean absolute
error by a factor of 3.6 over the best comparable baseline algorithm. Our
dataset and code are publicly available.

利用深度分类模型和序数回归损失优化学习自动对焦技术，相比之前的方法可以将平均绝对误差降低 3.6 倍。

自动对焦学习

Learning to Autofocus

In cinema, large camera lenses create beautiful shallow depth of field (DOF),
but make focusing difficult and expensive. Accurate cinema focus usually relies
on a script and a person to control focus in realtime. Casual videographers
often crave cinematic focus, but fail to achieve it. We either sacrifice
shallow DOF, as in smartphone videos; or we struggle to deliver accurate focus,
as in videos from larger cameras. This paper is about a new approach in the
pursuit of cinematic focus for casual videography. We present a system that
synthetically renders refocusable video from a deep DOF video shot with a
smartphone, and analyzes future video frames to deliver context-aware autofocus
for the current frame. To create refocusable video, we extend recent machine
learning methods designed for still photography, contributing a new dataset for
machine training, a rendering model better suited to cinema focus, and a
filtering solution for temporal coherence. To choose focus accurately for each
frame, we demonstrate autofocus that looks at upcoming video frames and applies
AI-assist modules such as motion, face, audio and saliency detection. We also
show that autofocus benefits from machine learning and a large-scale video
dataset with focus annotation, where we use our RVR-LAAF GUI to create this
sizable dataset efficiently. We deliver, for example, a shallow DOF video where
the autofocus transitions onto each person before she begins to speak. This is
impossible for conventional camera autofocus because it would require seeing
into the future.

本文介绍了一种新的方法来实现休闲视频制作中的电影般的焦点，即从深度拍摄的视频中合成可重新对焦的视频，并分析未来的视频帧以实现上下文感知的自动对焦。

针对非专业摄影师的合成模糊和前瞻自动对焦技术

Synthetic Defocus and Look-Ahead Autofocus for Casual Videography

This paper describes AutoFocus, an efficient multi-scale inference algorithm
for deep-learning based object detectors. Instead of processing an entire image
pyramid, AutoFocus adopts a coarse to fine approach and only processes regions
which are likely to contain small objects at finer scales. This is achieved by
predicting category agnostic segmentation maps for small objects at coarser
scales, called FocusPixels. FocusPixels can be predicted with high recall, and
in many cases, they only cover a small fraction of the entire image. To make
efficient use of FocusPixels, an algorithm is proposed which generates compact
rectangular FocusChips which enclose FocusPixels. The detector is only applied
inside FocusChips, which reduces computation while processing finer scales.
Different types of error can arise when detections from FocusChips of multiple
scales are combined, hence techniques to correct them are proposed. AutoFocus
obtains an mAP of 47.9% (68.3% at 50% overlap) on the COCO test-dev set while
processing 6.4 images per second on a Titan X (Pascal) GPU. This is 2.5X faster
than our multi-scale baseline detector and matches its mAP. The number of
pixels processed in the pyramid can be reduced by 5X with a 1% drop in mAP.
AutoFocus obtains more than 10% mAP gain compared to RetinaNet but runs at the
same speed with the same ResNet-101 backbone.

AutoFocus 是一种高效的基于深度学习的物体检测器的多尺度推理算法，只处理可能包含细节或小物体的区域，可以得到 47.9%（50% 重叠时的 68.3%）的 mAP，比 RetinaNet 多 10% 的 mAP。