This paper proposes a transformer over transformer framework, called
Transformer$^2$, to perform neural text segmentation. It consists of two
components: bottom-level sentence encoders using pre-trained transformers, and
an upper-level transformer-based segmentation model based on the sentence
embeddings. The bottom-level component transfers the pre-trained knowledge
learned from large external corpora under both single and pair-wise supervised
NLP tasks to model the sentence embeddings for the documents. Given the
sentence embeddings, the upper-level transformer is trained to recover the
segmentation boundaries as well as the topic labels of each sentence. Equipped
with a multi-task loss and the pre-trained knowledge, Transformer$^2$ can
better capture the semantic coherence within the same segments. Our experiments
show that (1) Transformer$^2$ manages to surpass state-of-the-art text
segmentation models in terms of a commonly-used semantic coherence measure; (2)
in most cases, both single and pair-wise pre-trained knowledge contribute to
the model performance; (3) bottom-level sentence encoders pre-trained on
specific languages yield better performance than those pre-trained on specific
domains.

本文提出了一种名为 Transformer$^2$ 的基于 transformer 的 transformer 框架，用于进行神经文本分割，并在实验中证明了其在语义一致性方面的优越性。

增强主题连贯性的基于预训练 Transformer 的神经文本分割模型

Transformer over Pre-trained Transformer for Neural Text Segmentation  with Enhanced Topic Coherence

Few-shot video classification aims to learn new video categories with only a
few labeled examples, alleviating the burden of costly annotation in real-world
applications. However, it is particularly challenging to learn a
class-invariant spatial-temporal representation in such a setting. To address
this, we propose a novel matching-based few-shot learning strategy for video
sequences in this work. Our main idea is to introduce an implicit temporal
alignment for a video pair, capable of estimating the similarity between them
in an accurate and robust manner. Moreover, we design an effective context
encoding module to incorporate spatial and feature channel context, resulting
in better modeling of intra-class variations. To train our model, we develop a
multi-task loss for learning video matching, leading to video features with
better generalization. Extensive experimental results on two challenging
benchmarks, show that our method outperforms the prior arts with a sizable
margin on SomethingSomething-V2 and competitive results on Kinetics.

本文提出了一种新的基于匹配的少样本学习策略，结合隐式时间对齐和上下文编码模块以及多任务损失，用于解决视频分类中在有限标注情况下空间 - 时间表征的学习问题，并在两个具有挑战性的基准测试中证明了其优越性。