Sample contrastive methods, typically referred to simply as contrastive are
the foundation of most unsupervised methods to learn text and sentence
embeddings. On the other hand, a different class of self-supervised loss
functions and methods have been considered in the computer vision community and
referred to as dimension contrastive. In this paper, we thoroughly compare this
class of methods with the standard baseline for contrastive sentence
embeddings, SimCSE. We find that self-supervised embeddings trained using
dimension contrastive objectives can outperform SimCSE on downstream tasks
without needing auxiliary loss functions.

通过比较对比句子嵌入方法中的标准基准 SimCSE 与计算机视觉社区中被称为维度对比的自监督损失函数和方法，我们发现使用维度对比目标训练的自监督嵌入能够在下游任务中胜过 SimCSE 而无需辅助损失函数。

基于自监督学习的非对比句子表示

Non-contrastive sentence representations via self-supervision

Given an untrimmed video, temporal sentence localization (TSL) aims to
localize a specific segment according to a given sentence query. Though
respectable works have made decent achievements in this task, they severely
rely on dense video frame annotations, which require a tremendous amount of
human effort to collect. In this paper, we target another more practical and
challenging setting: one-shot temporal sentence localization (one-shot TSL),
which learns to retrieve the query information among the entire video with only
one annotated frame. Particularly, we propose an effective and novel
tree-structure baseline for one-shot TSL, called Multiple Hypotheses Segment
Tree (MHST), to capture the query-aware discriminative frame-wise information
under the insufficient annotations. Each video frame is taken as the leaf-node,
and the adjacent frames sharing the same visual-linguistic semantics will be
merged into the upper non-leaf node for tree building. At last, each root node
is an individual segment hypothesis containing the consecutive frames of its
leaf-nodes. During the tree construction, we also introduce a pruning strategy
to eliminate the interference of query-irrelevant nodes. With our designed
self-supervised loss functions, our MHST is able to generate high-quality
segment hypotheses for ranking and selection with the query. Experiments on two
challenging datasets demonstrate that MHST achieves competitive performance
compared to existing methods.

本文提出了一种基于叶子节点为每个视频帧的多假设片段树（MHST）的方法，从而解决在极少的帧注释下，实现单次时序句子本地化（one-shot TSL）的问题，并提供了自监督损失函数，与现有方法相比取得了具有竞争力的性能。

一次性时间句子定位的假设树构建

Hypotheses Tree Building for One-Shot Temporal Sentence Localization

Logical reasoning of text is an important ability that requires understanding
the information present in the text, their interconnections, and then reasoning
through them to infer new conclusions. Prior works on improving the logical
reasoning ability of language models require complex processing of training
data (e.g., aligning symbolic knowledge to text), yielding task-specific data
augmentation solutions that restrict the learning of general logical reasoning
skills. In this work, we propose APOLLO, an adaptively pretrained language
model that has improved logical reasoning abilities. We select a subset of
Wikipedia, based on a set of logical inference keywords, for continued
pretraining of a language model. We use two self-supervised loss functions: a
modified masked language modeling loss where only specific parts-of-speech
words, that would likely require more reasoning than basic language
understanding, are masked, and a sentence-level classification loss that
teaches the model to distinguish between entailment and contradiction types of
sentences. The proposed training paradigm is both simple and independent of
task formats. We demonstrate the effectiveness of APOLLO by comparing it with
prior baselines on two logical reasoning datasets. APOLLO performs comparably
on ReClor and outperforms baselines on LogiQA.

本文提出了一个适应性预训练语言模型 APOLLO，使用特定的逻辑推理关键词来选择维基百科的子集进行预训练，使用自我监督损失函数进行训练，无需特定的任务格式，与基线模型相比，在逻辑推理数据集上表现相当或更好。