Owing to success in the data-rich domain of natural images, Transformers have
recently become popular in medical image segmentation. However, the pairing of
Transformers with convolutional blocks in varying architectural permutations
leaves their relative effectiveness to open interpretation. We introduce
Transformer Ablations that replace the Transformer blocks with plain linear
operators to quantify this effectiveness. With experiments on 8 models on 2
medical image segmentation tasks, we explore -- 1) the replaceable nature of
Transformer-learnt representations, 2) Transformer capacity alone cannot
prevent representational replaceability and works in tandem with effective
design, 3) The mere existence of explicit feature hierarchies in transformer
blocks is more beneficial than accompanying self-attention modules, 4) Major
spatial downsampling before Transformer modules should be used with caution.

通过使用线性算子替换 Transformer 块，量化了 Transformer 在医学图像分割中的效果，实验证明显式特征层次在 Transformer 块中的存在比自我关注模块更有益，但在 Transformer 模块之前应谨慎使用主要的空间下采样。

医学图像分割网络中的 Transformer 利用

Transformer Utilization in Medical Image Segmentation Networks

Recent years have seen the emergence of pre-trained representations as a
powerful abstraction for AI applications in computer vision, natural language,
and speech. However, policy learning for control is still dominated by a
tabula-rasa learning paradigm, with visuo-motor policies often trained from
scratch using data from deployment environments. In this context, we revisit
and study the role of pre-trained visual representations for control, and in
particular representations trained on large-scale computer vision datasets.
Through extensive empirical evaluation in diverse control domains (Habitat,
DeepMind Control, Adroit, Franka Kitchen), we isolate and study the importance
of different representation training methods, data augmentations, and feature
hierarchies. Overall, we find that pre-trained visual representations can be
competitive or even better than ground-truth state representations to train
control policies. This is in spite of using only out-of-domain data from
standard vision datasets, without any in-domain data from the deployment
environments. Source code and more at
this https URL.

研究表明，预训练的视觉表示方法对于控制任务的学习同样有效，甚至在某些情况下可以优于基于真实状态的表示方法，而这些预训练模型仅需要使用标准视觉数据集，而不需要在目标环境中使用真实数据。