Transformer is leading a trend in the field of image processing. Despite the
great success that existing lightweight image processing transformers have
achieved, they are tailored to FLOPs or parameters reduction, rather than
practical inference acceleration. In this paper, we present a latency-aware
image processing transformer, termed LIPT. We devise the low-latency proportion
LIPT block that substitutes memory-intensive operators with the combination of
self-attention and convolutions to achieve practical speedup. Specifically, we
propose a novel non-volatile sparse masking self-attention (NVSM-SA) that
utilizes a pre-computing sparse mask to capture contextual information from a
larger window with no extra computation overload. Besides, a high-frequency
reparameterization module (HRM) is proposed to make LIPT block
reparameterization friendly, which improves the model's detail reconstruction
capability. Extensive experiments on multiple image processing tasks (e.g.,
image super-resolution (SR), JPEG artifact reduction, and image denoising)
demonstrate the superiority of LIPT on both latency and PSNR. LIPT achieves
real-time GPU inference with state-of-the-art performance on multiple image SR
benchmarks.

本文提出了一种低延迟的图像处理 transformer，名为 LIPT，通过使用自注意力和卷积的组合来替代内存密集型运算符，以实现实用的加速。在多个图像处理任务上进行的广泛实验表明，LIPT 在延迟和 PSNR 方面均表现出优越性，实现了多个图像超分辨率基准测试上的实时 GPU 推理和最先进的性能。

LIPT：时延感知图像处理 Transformer

LIPT: Latency-aware Image Processing Transformer

Deep learning models have achieved expert-level performance in healthcare
with an exclusive focus on training accurate models. However, in many clinical
environments such as intensive care unit (ICU), real-time model serving is
equally if not more important than accuracy, because in ICU patient care is
simultaneously more urgent and more expensive. Clinical decisions and their
timeliness, therefore, directly affect both the patient outcome and the cost of
care. To make timely decisions, we argue the underlying serving system must be
latency-aware. To compound the challenge, health analytic applications often
require a combination of models instead of a single model, to better specialize
individual models for different targets, multi-modal data, different prediction
windows, and potentially personalized predictions. To address these challenges,
we propose HOLMES-an online model ensemble serving framework for healthcare
applications. HOLMES dynamically identifies the best performing set of models
to ensemble for highest accuracy, while also satisfying sub-second latency
constraints on end-to-end prediction. We demonstrate that HOLMES is able to
navigate the accuracy/latency tradeoff efficiently, compose the ensemble, and
serve the model ensemble pipeline, scaling to simultaneously streaming data
from 100 patients, each producing waveform data at 250~Hz. HOLMES outperforms
the conventional offline batch-processed inference for the same clinical task
in terms of accuracy and latency (by order of magnitude). HOLMES is tested on
risk prediction task on pediatric cardio ICU data with above 95% prediction
accuracy and sub-second latency on 64-bed simulation.

本文提出了一种在线模型集成服务框架 (HOLMES) 来应对医疗应用中的实时决策需求和准确性需求，在实现高精度的同时满足亚秒级的端到端预测延迟限制，通过测试证明其在儿童心脏 ICU 数据的风险预测任务上能够达到 95% 以上的预测准确率和亚秒级的预测延迟。

HOLMES: 重症监护室深度学习模型的健康在线模型集成服务

HOLMES: Health OnLine Model Ensemble Serving for Deep Learning Models in  Intensive Care Units

Despite the blooming success of architecture search for vision tasks in
resource-constrained environments, the design of on-device object detection
architectures have mostly been manual. The few automated search efforts are
either centered around non-mobile-friendly search spaces or not guided by
on-device latency. We propose MnasFPN, a mobile-friendly search space for the
detection head, and combine it with latency-aware architecture search to
produce efficient object detection models. The learned MnasFPN head, when
paired with MobileNetV2 body, outperforms MobileNetV3+SSDLite by 1.8 mAP at
similar latency on Pixel. It is also both 1.0 mAP more accurate and 10% faster
than NAS-FPNLite. Ablation studies show that the majority of the performance
gain comes from innovations in the search space. Further explorations reveal an
interesting coupling between the search space design and the search algorithm,
and that the complexity of MnasFPN search space may be at a local optimum.

本研究提出 MnasFPN 架构搜索空间以及结合基于延迟感知的搜索方法的对象检测模型设计，与 MobileNetV2 体系结构相匹配，优于 MobileNetV3+SSDLite 1.8 mAP，在与 NAS-FPNLite 相比，精度更高 1.0 mAP，速度更快 10%。