In this paper, we develop a MultiTask Learning (MTL) model to achieve dense
predictions for comics panels to, in turn, facilitate the transfer of comics
from one publication channel to another by assisting authors in the task of
reconfiguring their narratives. Our MTL method can successfully identify the
semantic units as well as the embedded notion of 3D in comic panels. This is a
significantly challenging problem because comics comprise disparate artistic
styles, illustrations, layouts, and object scales that depend on the authors
creative process. Typically, dense image-based prediction techniques require a
large corpus of data. Finding an automated solution for dense prediction in the
comics domain, therefore, becomes more difficult with the lack of ground-truth
dense annotations for the comics images. To address these challenges, we
develop the following solutions: 1) we leverage a commonly-used strategy known
as unsupervised image-to-image translation, which allows us to utilize a large
corpus of real-world annotations; 2) we utilize the results of the translations
to develop our multitasking approach that is based on a vision transformer
backbone and a domain transferable attention module; 3) we study the
feasibility of integrating our MTL dense-prediction method with an existing
retargeting method, thereby reconfiguring comics.

本文提出了一种 MTL 模型，以实现对漫画画板的密集预测，从而帮助作者重新组织他们的叙述，并探讨了与现有方法整合的可行性。

密集多任务学习以重新配置连环漫画

Dense Multitask Learning to Reconfigure Comics

In this work, we propose a novel and scalable solution to address the
challenges of developing efficient dense predictions on edge platforms. Our
first key insight is that MultiTask Learning (MTL) and hardware-aware Neural
Architecture Search (NAS) can work in synergy to greatly benefit on-device
Dense Predictions (DP). Empirical results reveal that the joint learning of the
two paradigms is surprisingly effective at improving DP accuracy, achieving
superior performance over both the transfer learning of single-task NAS and
prior state-of-the-art approaches in MTL, all with just 1/10th of the
computation. To the best of our knowledge, our framework, named EDNAS, is the
first to successfully leverage the synergistic relationship of NAS and MTL for
DP. Our second key insight is that the standard depth training for multi-task
DP can cause significant instability and noise to MTL evaluation. Instead, we
propose JAReD, an improved, easy-to-adopt Joint Absolute-Relative Depth loss,
that reduces up to 88% of the undesired noise while simultaneously boosting
accuracy. We conduct extensive evaluations on standard datasets, benchmark
against strong baselines and state-of-the-art approaches, as well as provide an
analysis of the discovered optimal architectures.

该论文提出了一种名为 EDNAS 的新型架构，通过多任务学习（MTL）和硬件感知神经架构搜索（NAS）的联合学习，成功地提高了边缘平台上稠密预测的准确性，并通过提出 JAReD 方法降低了输出噪声，并揭示了最佳架构。

面向边缘端的协同多任务神经结构搜索优化稠密预测性能

Toward Edge-Efficient Dense Predictions with Synergistic Multi-Task Neural Architecture Search

Recently, there emerges a series of vision Transformers, which show superior
performance with a more compact model size than conventional convolutional
neural networks, thanks to the strong ability of Transformers to model
long-range dependencies. However, the advantages of vision Transformers also
come with a price: Self-attention, the core part of Transformer, has a
quadratic complexity to the input sequence length. This leads to a dramatic
increase of computation and memory cost with the increase of sequence length,
thus introducing difficulties when applying Transformers to the vision tasks
that require dense predictions based on high-resolution feature maps. In this
paper, we propose a new vision Transformer, named Glance-and-Gaze Transformer
(GG-Transformer), to address the aforementioned issues. It is motivated by the
Glance and Gaze behavior of human beings when recognizing objects in natural
scenes, with the ability to efficiently model both long-range dependencies and
local context. In GG-Transformer, the Glance and Gaze behavior is realized by
two parallel branches: The Glance branch is achieved by performing
self-attention on the adaptively-dilated partitions of the input, which leads
to a linear complexity while still enjoying a global receptive field; The Gaze
branch is implemented by a simple depth-wise convolutional layer, which
compensates local image context to the features obtained by the Glance
mechanism. We empirically demonstrate our method achieves consistently superior
performance over previous state-of-the-art Transformers on various vision tasks
and benchmarks. The codes and models will be made available at
this https URL

提出一种效率更高的视觉 Transformer 模型，命名为 Glance-and-Gaze Transformer (GG-Transformer)，其通过两个并行的分支 ——Glance 和 Gaze，分别实现自适应扩张分区的自注意力机制与简单深度卷积层的局部图像上下文补偿，从而实现长距离依赖性和局部区域信息的高效建模，在多个视觉任务和基准测试中都表现出了优秀的性能。

速览注视视觉变换器

Glance-and-Gaze Vision Transformer

Temporal action localization is an important task of computer vision. Though
many methods have been proposed, it still remains an open question how to
predict the temporal location of action segments precisely. Most
state-of-the-art works train action classifiers on video segments
pre-determined by action proposal. However, recent work found that a desirable
model should move beyond segment-level and make dense predictions at a fine
granularity in time to determine precise temporal boundaries. In this paper, we
propose a Frame Segmentation Network (FSN) that places a temporal CNN on top of
the 2D spatial CNNs. Spatial CNNs are responsible for abstracting semantics in
spatial dimension while temporal CNN is responsible for introducing temporal
context information and performing dense predictions. The proposed FSN can make
dense predictions at frame-level for a video clip using both spatial and
temporal context information. FSN is trained in an end-to-end manner, so the
model can be optimized in spatial and temporal domain jointly. We also adapt
FSN to use it in weakly supervised scenario (WFSN), where only video level
labels are provided when training. Experiment results on public dataset show
that FSN achieves superior performance in both frame-level action localization
and temporal action localization.

本研究提出了一种基于帧分割网络（FSN）的方法，通过在 2D 空间 CNN 上放置时间 CNN，利用时空上下文信息通过帧级密集预测，实现对视频剪辑的精确定位，该方法可在弱监督场景下使用，实现了优越的帧级动作定位和时间上的动作定位。