Early neural network models relied exclusively on bottom-up processing going
from the input signals to higher-level representations. Many recent models also
incorporate top-down networks going in the opposite direction. Top-down
processing in deep learning models plays two primary roles: learning and
directing attention. These two roles are accomplished in current models through
distinct mechanisms. While top-down attention is often implemented by extending
the model's architecture with additional units that propagate information from
high to low levels of the network, learning is typically accomplished by an
external learning algorithm such as back-propagation. In the current work, we
present an integration of the two functions above, which appear unrelated,
using a single unified mechanism. We propose a novel symmetric bottom-up
top-down network structure that can integrate standard bottom-up networks with
a symmetric top-down counterpart, allowing each network to guide and influence
the other. The same top-down network is being used for both learning, via
back-propagating feedback signals, and at the same time also for top-down
attention, by guiding the bottom-up network to perform a selected task. We show
that our method achieves competitive performance on a standard multi-task
learning benchmark. Yet, we rely on standard single-task architectures and
optimizers, without any task-specific parameters. Additionally, our learning
algorithm addresses in a new way some neuroscience issues that arise in
biological modeling of learning in the brain.

本研究提出了一种新的神经网络结构，它将底部网络和对称的顶部网络相结合，通过单一的统一机制实现了学习和引导注意力等两种主要功能。该模型在多任务学习基准测试中表现出竞争力，并且还能够解决一些在生物学建模中出现的神经科学问题。

自顶向下处理：自顶向下网络将反向传播与注意力结合

Top-Down Processing: Top-Down Network Combines Back-Propagation with  Attention

How to best integrate linguistic and perceptual processing in multi-modal
tasks that involve language and vision is an important open problem. In this
work, we argue that the common practice of using language in a top-down manner,
to direct visual attention over high-level visual features, may not be optimal.
We hypothesize that the use of language to also condition the bottom-up
processing from pixels to high-level features can provide benefits to the
overall performance. To support our claim, we propose a U-Net-based model and
perform experiments on two language-vision dense-prediction tasks: referring
expression segmentation and language-guided image colorization. We compare
results where either one or both of the top-down and bottom-up visual branches
are conditioned on language. Our experiments reveal that using language to
control the filters for bottom-up visual processing in addition to top-down
attention leads to better results on both tasks and achieves competitive
performance. Our linguistic analysis suggests that bottom-up conditioning
improves segmentation of objects especially when input text refers to low-level
visual concepts. Code is available at this https URL

本文旨在探讨如何在涉及语言和视觉的多模态任务中最佳地整合语言和感知处理，并提出了使用语言来调节自下而上处理以及自上而下注意力的 U-Net 模型。 实验结果表明，使用语言控制自下而上的视觉处理可提高多任务的细分结果表现。