The rapid scaling of large vision pretrained models makes fine-tuning tasks
more and more difficult on edge devices with low computational resources. We
explore a new visual adaptation paradigm called edge tuning, which treats large
pretrained models as standalone feature extractors that run on powerful cloud
servers. The fine-tuning carries out on edge devices with small networks which
require low computational resources. Existing methods that are potentially
suitable for our edge tuning paradigm are discussed. But, three major drawbacks
hinder their application in edge tuning: low adaptation capability, large
adapter network, and high information transfer overhead. To address these
issues, we propose Minimal Interaction Edge Tuning, or MIET, which reveals that
the sum of intermediate features from pretrained models not only has minimal
information transfer but also has high adaptation capability. With a
lightweight attention-based adaptor network, MIET achieves information transfer
efficiency, parameter efficiency, computational and memory efficiency, and at
the same time demonstrates competitive results on various visual adaptation
benchmarks.

MIET（极简交互边缘调优）通过使用轻量级的基于注意力机制的适配器网络，利用预训练模型的中间特征的加和实现了信息传递效率、参数效率、计算和内存效率，同时在各种视觉适应测试中展示出有竞争力的结果。

极简交互边缘调校：视觉适应的新范式

Minimal Interaction Edge Tuning: A New Paradigm for Visual Adaptation

Natural language rationales could provide intuitive, higher-level
explanations that are easily understandable by humans, complementing the more
broadly studied lower-level explanations based on gradients or attention
weights. We present the first study focused on generating natural language
rationales across several complex visual reasoning tasks: visual commonsense
reasoning, visual-textual entailment, and visual question answering. The key
challenge of accurate rationalization is comprehensive image understanding at
all levels: not just their explicit content at the pixel level, but their
contextual contents at the semantic and pragmatic levels. We present
Rationale^VT Transformer, an integrated model that learns to generate free-text
rationales by combining pretrained language models with object recognition,
grounded visual semantic frames, and visual commonsense graphs. Our experiments
show that the base pretrained language model benefits from visual adaptation
and that free-text rationalization is a promising research direction to
complement model interpretability for complex visual-textual reasoning tasks.

本文介绍了首个以生成自然语言解释为中心的研究，该研究关注复杂的视觉推理任务，包括视觉常识推理、视觉文本蕴含和视觉问答。文章提出了 Rationale^VT Transformer，该模型通过结合预训练的语言模型、对象识别、基于视觉的语义框架和视觉常识图生成自由文本解释，实现了全面的图像理解，并且实验证明，自然语言解释是用于复杂视觉 - 文本推理任务的一种具有前景的研究方向。

从像素到语义框架到常识图谱的全栈视觉推理自然语言理由

Natural Language Rationales with Full-Stack Visual Reasoning: From  Pixels to Semantic Frames to Commonsense Graphs

Domain adaptation from one data space (or domain) to another is one of the
most challenging tasks of modern data analytics. If the adaptation is done
correctly, models built on a specific data space become more robust when
confronted to data depicting the same semantic concepts (the classes), but
observed by another observation system with its own specificities. Among the
many strategies proposed to adapt a domain to another, finding a common
representation has shown excellent properties: by finding a common
representation for both domains, a single classifier can be effective in both
and use labelled samples from the source domain to predict the unlabelled
samples of the target domain. In this paper, we propose a regularized
unsupervised optimal transportation model to perform the alignment of the
representations in the source and target domains. We learn a transportation
plan matching both PDFs, which constrains labelled samples in the source domain
to remain close during transport. This way, we exploit at the same time the few
labeled information in the source and the unlabelled distributions observed in
both domains. Experiments in toy and challenging real visual adaptation
examples show the interest of the method, that consistently outperforms state
of the art approaches.

该研究提出了一种规范化的非监督最优运输模型，通过找到公共表示来在源域和目标域之间执行两个领域的对齐，从而在标记和未标记的数据中同时利用了源和目标领域中的信息，该模型在可视化自适应方面表现优异。