In this work, we have worked towards two major goals. Firstly, we have
investigated the importance of Batch Normalisation (BN) layers in a
non-contrastive representation learning framework called Bootstrap Your Own
Latent (BYOL). We conducted several experiments to conclude that BN layers are
not necessary for representation learning in BYOL. Moreover, BYOL only learns
from the positive pairs of images but ignores other semantically similar images
in the same input batch. For the second goal, we have introduced two new loss
functions to determine the semantically similar pairs in the same input batch
of images and reduce the distance between their representations. These loss
functions are Cross-Cosine Similarity Loss (CCSL) and Cross-Sigmoid Similarity
Loss (CSSL). Using the proposed loss functions, we are able to surpass the
performance of Vanilla BYOL (71.04%) by training the BYOL framework using CCSL
loss (76.87%) on the STL10 dataset. BYOL trained using CSSL loss performs
comparably with Vanilla BYOL.

研究 BYOL 中批标准化层的重要性，提出使用 CCSL 和 CSSL 进行相似图片对比的方法来改进 BYOL 的表征学习效果。在 STL10 数据集上使用 CSSL loss 训练 BYOL 框架可以取得较好的效果。

基于自举潜变量的强鲁棒性表示学习的自标注优化

Self-Labeling Refinement for Robust Representation Learning with Bootstrap Your Own Latent

Transfer learning from ImageNet is the go-to approach when applying deep
learning to medical images. The approach is either to fine-tune a pre-trained
model or use it as a feature extractor. Most modern architecture contain batch
normalisation layers, and fine-tuning a model with such layers requires taking
a few precautions as they consist of trainable and non-trainable weights and
have two operating modes: training and inference. Attention is primarily given
to the non-trainable weights used during inference, as they are the primary
source of unexpected behaviour or degradation in performance during transfer
learning. It is typically recommended to fine-tune the model with the batch
normalisation layers kept in inference mode during both training and inference.
In this paper, we pay closer attention instead to the trainable weights of the
batch normalisation layers, and we explore their expressive influence in the
context of transfer learning. We find that only fine-tuning the trainable
weights (scale and centre) of the batch normalisation layers leads to similar
performance as to fine-tuning all of the weights, with the added benefit of
faster convergence. We demonstrate this on a variety of seven publicly
available medical imaging datasets, using four different model architectures.

本文探究了深度学习在医学图像上的转移学习技术，着重研究了 Batch Normalisation 层的可训练权重在 fine-tune 过程中的表现并发现只对其可训练权重 fine-tune 可以提高模型性能和收敛速度。

可训练 Batch Norm 参数对迁移学习的表现影响

Partial transfusion: on the expressive influence of trainable batch norm  parameters for transfer learning

Owing to flexible architectures of deep convolutional neural networks (CNNs),
CNNs are successfully used for image denoising. However, they suffer from the
following drawbacks: (i) deep network architecture is very difficult to train.
(ii) Deeper networks face the challenge of performance saturation. In this
study, the authors propose a novel method called enhanced convolutional neural
denoising network (ECNDNet). Specifically, they use residual learning and batch
normalisation techniques to address the problem of training difficulties and
accelerate the convergence of the network. In addition, dilated convolutions
are used in the proposed network to enlarge the context information and reduce
the computational cost. Extensive experiments demonstrate that the ECNDNet
outperforms the state-of-the-art methods for image denoising.

本文提出了一种名为增强卷积神经去噪网络（ECNDNet）的新方法，通过使用残差学习和批量归一化技术提高了训练效果并加速了网络的收敛。此外，提议的网络使用扩张卷积增大了上下文信息，降低了计算成本，并在广泛实验中证明了 ECNDNet 优于图像去噪的现有最先进方法。