We introduce sub-sentence encoder, a contrastively-learned contextual
embedding model for fine-grained semantic representation of text. In contrast
to the standard practice with sentence embeddings, where the meaning of an
entire sequence of text is encoded into a fixed-length vector, the sub-sentence
encoder learns to produce distinct contextual embeddings corresponding to
different atomic propositions, i.e. atomic units of meaning expressed within a
text sequence. The sub-sentence embeddings are contrastively learned to
recognize (inferred) semantic equivalence between propositions across different
text sequences. Our experiments show the effectiveness of sub-sentence encoders
in applications, such as retrieving supporting facts for fine-grained text
attribution or recognizing the conditional semantic similarity between texts.
In practice, we demonstrate that sub-sentence encoders keep the same level of
inference cost and space complexity compared to sentence encoders.

引入一种次句编码器，一种对文本进行细粒度语义表示的对比学习上下文嵌入模型。与对整个文本序列进行编码为固定长度向量的标准实践不同，次句编码器学习产生与文本序列中不同的原子命题（即表达在一个文本序列中的含义的原子单位）相对应的不同的上下文嵌入。次句嵌入通过对比学习来识别（推导出的）命题在不同文本序列中的语义等价性。我们的实验表明，次句编码器在应用中的有效性，例如检索细粒度文本归因的支持事实或识别文本之间的条件语义相似性。在实践中，我们证明次句编码器与句子编码器相比具有相同的推理成本和空间复杂度。

子句编码器：命题语义表征的对比学习

Sub-Sentence Encoder: Contrastive Learning of Propositional Semantic  Representations

Discriminative learning, restorative learning, and adversarial learning have
proven beneficial for self-supervised learning schemes in computer vision and
medical imaging. Existing efforts, however, omit their synergistic effects on
each other in a ternary setup, which, we envision, can significantly benefit
deep semantic representation learning. To realize this vision, we have
developed DiRA, the first framework that unites discriminative, restorative,
and adversarial learning in a unified manner to collaboratively glean
complementary visual information from unlabeled medical images for fine-grained
semantic representation learning. Our extensive experiments demonstrate that
DiRA (1) encourages collaborative learning among three learning ingredients,
resulting in more generalizable representation across organs, diseases, and
modalities; (2) outperforms fully supervised ImageNet models and increases
robustness in small data regimes, reducing annotation cost across multiple
medical imaging applications; (3) learns fine-grained semantic representation,
facilitating accurate lesion localization with only image-level annotation; and
(4) enhances state-of-the-art restorative approaches, revealing that DiRA is a
general mechanism for united representation learning. All code and pre-trained
models are available at https: //github.com/JLiangLab/DiRA.

DiRA 是第一个将区分性、修复性、对抗性学习统一起来协同获取未标记医学图像的互补视觉信息，用于精细语义表示学习的框架，具有更强的泛化能力，超越了完全监督的 ImageNet 模型并提高了在小数据情况下的鲁棒性，是一种联合表示学习的通用机制。