Recently, self-supervised learning (SSL) methods have been used in
pre-training the segmentation models for 2D and 3D medical images. Most of
these methods are based on reconstruction, contrastive learning and consistency
regularization. However, the spatial correspondence of 2D slices from a 3D
medical image has not been fully exploited. In this paper, we propose a novel
self-supervised alignment learning framework to pre-train the neural network
for medical image segmentation. The proposed framework consists of a new local
alignment loss and a global positional loss. We observe that in the same 3D
scan, two close 2D slices usually contain similar anatomic structures. Thus,
the local alignment loss is proposed to make the pixel-level features of
matched structures close to each other. Experimental results show that the
proposed alignment learning is competitive with existing self-supervised
pre-training approaches on CT and MRI datasets, under the setting of limited
annotations.

最近，自监督学习方法已被用于对二维和三维医学图像的分割模型进行预训练。本文提出了一种新的自监督对齐学习框架来预训练医学图像分割的神经网络，其中包括了一个全新的局部对齐损失和全局位置损失。实验结果表明，该对齐学习方法在有限标注的 CT 和 MRI 数据集上与现有的自监督预训练方法具有竞争力。

医学图像分割的自我监督对齐学习

Self-Supervised Alignment Learning for Medical Image Segmentation

Aligning large language models (LLMs) to human values has become increasingly
important as it enables sophisticated steering of LLMs, e.g., making them
follow given instructions while keeping them less toxic. However, it requires a
significant amount of human demonstrations and feedback. Recently, open-sourced
models have attempted to replicate the alignment learning process by distilling
data from already aligned LLMs like InstructGPT or ChatGPT. While this process
reduces human efforts, constructing these datasets has a heavy dependency on
the teacher models. In this work, we propose a novel framework for alignment
learning with almost no human labor and no dependency on pre-aligned LLMs.
First, we perform reward modeling (RM) with synthetic feedback by contrasting
responses from vanilla LLMs with various sizes and prompts. Then, we use the RM
for simulating high-quality demonstrations to train a supervised policy and for
further optimizing the model with reinforcement learning. Our resulting model,
Aligned Language Model with Synthetic Training dataset (ALMoST), outperforms
open-sourced models, including Alpaca, Dolly, and OpenAssistant, which are
trained on the outputs of InstructGPT or human-annotated instructions. Our
7B-sized model outperforms the 12-13B models in the A/B tests using GPT-4 as
the judge with about 75% winning rate on average.

本研究提出了一个新的框架，利用奖励建模 (RM) 方法和模拟高质量演示来进行对齐语言模型的训练，避免了对已对齐的 LLMs 的依赖，这种方法的结果是，我们的模型 ALMoST 在对 InstructGPT 或人工注释指令训练的开放源代码模型中表现良好，我们的 7B 大小的模型在使用 GPT-4 作为评判员的 A /B 测试中表现优异，平均获胜率约为 75％。

通过合成反馈对齐大型语言模型

Aligning Large Language Models through Synthetic Feedback

Attention mechanism in sequence-to-sequence models is designed to model the
alignments between acoustic features and output tokens in speech recognition.
However, attention weights produced by models trained end to end do not always
correspond well with actual alignments, and several studies have further argued
that attention weights might not even correspond well with the relevance
attribution of frames. Regardless, visual similarity between attention weights
and alignments is widely used during training as an indicator of the models
quality. In this paper, we treat the correspondence between attention weights
and alignments as a learning problem by imposing a supervised attention loss.
Experiments have shown significant improved performance, suggesting that
learning the alignments well during training critically determines the
performance of sequence-to-sequence models.

本文研究序列到序列模型中注意力机制，通过监督学习的方法对注意力权重和输出对齐进行训练，实验结果表明，对齐学习对模型性能的提升至关重要。