Recently, Transformers have shown promising performance in various vision
tasks. To reduce the quadratic computation complexity caused by each query
attending to all keys/values, various methods have constrained the range of
attention within local regions, where each query only attends to keys/values
within a hand-crafted window. However, these hand-crafted window partition
mechanisms are data-agnostic and ignore their input content, so it is likely
that one query maybe attends to irrelevant keys/values. To address this issue,
we propose a Dynamic Group Attention (DG-Attention), which dynamically divides
all queries into multiple groups and selects the most relevant keys/values for
each group. Our DG-Attention can flexibly model more relevant dependencies
without any spatial constraint that is used in hand-crafted window based
attention. Built on the DG-Attention, we develop a general vision transformer
backbone named Dynamic Group Transformer (DGT). Extensive experiments show that
our models can outperform the state-of-the-art methods on multiple common
vision tasks, including image classification, semantic segmentation, object
detection, and instance segmentation.

该论文提出了一种动态分组注意力 (DG-Attention) 和一个通用的 DGT 视觉转换器的背景，可以在多个常见的视觉任务中超越现有技术，并动态地将所有查询分成多个组，为每个组选择最相关的密钥 / 值。

动态分组变换器：具有动态分组注意力的通用视觉变换器骨干网络

Dynamic Group Transformer: A General Vision Transformer Backbone with  Dynamic Group Attention

This document describes the findings of the Third Workshop on Neural
Generation and Translation, held in concert with the annual conference of the
Empirical Methods in Natural Language Processing (EMNLP 2019). First, we
summarize the research trends of papers presented in the proceedings. Second,
we describe the results of the two shared tasks 1) efficient neural machine
translation (NMT) where participants were tasked with creating NMT systems that
are both accurate and efficient, and 2) document-level generation and
translation (DGT) where participants were tasked with developing systems that
generate summaries from structured data, potentially with assistance from text
in another language.

本文概述了第三届神经网络生成和翻译研讨会的研究趋势，介绍了两项共享任务的结果，即提高神经机器翻译的效率以及文档级生成和翻译。