Taking inspiration from Set Theory, we introduce SetCSE, an innovative
information retrieval framework. SetCSE employs sets to represent complex
semantics and incorporates well-defined operations for structured information
querying under the provided context. Within this framework, we introduce an
inter-set contrastive learning objective to enhance comprehension of sentence
embedding models concerning the given semantics. Furthermore, we present a
suite of operations, including SetCSE intersection, difference, and operation
series, that leverage sentence embeddings of the enhanced model for complex
sentence retrieval tasks. Throughout this paper, we demonstrate that SetCSE
adheres to the conventions of human language expressions regarding compounded
semantics, provides a significant enhancement in the discriminatory capability
of underlying sentence embedding models, and enables numerous information
retrieval tasks involving convoluted and intricate prompts which cannot be
achieved using existing querying methods.

通过引用集合理论，我们提出了 SetCSE，一种创新的信息检索框架。SetCSE 使用集合来表示复杂语义，并结合提供的上下文进行结构化信息查询。在该框架中，我们引入了一种集合间对比学习目标，以增强对给定语义的句子嵌入模型的理解。此外，我们提供了一系列操作，包括 SetCSE 的交集、差集和操作系列，以利用增强模型的句子嵌入进行复杂句子检索任务。本文中，我们证明 SetCSE 遵循人类语言表达的约定，显著提高了底层句子嵌入模型的区分能力，并能够通过使用现有查询方法无法实现的复杂检索任务。

SetCSE：使用对比学习的句子嵌入进行集合操作

SetCSE: Set Operations using Contrastive Learning of Sentence Embeddings

Modeling long texts has been an essential technique in the field of natural
language processing (NLP). With the ever-growing number of long documents, it
is important to develop effective modeling methods that can process and analyze
such texts. However, long texts pose important research challenges for existing
text models, with more complex semantics and special characteristics. In this
paper, we provide an overview of the recent advances on long texts modeling
based on Transformer models. Firstly, we introduce the formal definition of
long text modeling. Then, as the core content, we discuss how to process long
input to satisfy the length limitation and design improved Transformer
architectures to effectively extend the maximum context length. Following this,
we discuss how to adapt Transformer models to capture the special
characteristics of long texts. Finally, we describe four typical applications
involving long text modeling and conclude this paper with a discussion of
future directions. Our survey intends to provide researchers with a synthesis
and pointer to related work on long text modeling.

本篇论文综述了最新的基于 Transformer 模型的长文本建模技术，其中介绍了长文本模型的定义、如何满足输入长度限制和改进 Transformer 的架构以有效扩展最大上下文长度，以及如何适应长文本的特殊特性。并且描述了长文本建模的四个典型应用，并探讨了未来的研究方向。